LetsEncrypt, CDK 및 AWS Lambda를 사용하여 ECS Fargate 컨테이너에 직접 액세스(나쁜 생각일 수 있음)
88139 단어 devopsserverlessawstypescript
${taskid}.eu-west-2.browser.reflow.io
.이것은 거의 항상 나쁜 생각이다.그러나 만약 당신이 정말 이렇게 해야 한다면, 이 지침은 도움이 될 것이다.
너는 왜 이렇게 하지 말았어야 했니
AWS는 컨테이너화된 작업 부하를 조작하기 위해 다양한 전투 테스트를 거친 모델을 제공한다.이러한 모델은 더욱 쉽게 배치되고 파괴될 수 없으며 절대 다수의 고객 용례에 적용된다.예를 들면 다음과 같습니다.
Application Load Balanced Fargate Service: ECS 클러스터에서 실행되는 Fargate 서비스는 어플리케이션 로드 밸런서에 의해 제공됩니다.이점:
Queue Processing Fargate Service: SQS 대기열의 작업을 처리하기 위해 자동으로 확장되는Fargate 서비스입니다.이점:
우리 왜 이러는지
reflow에서 웹 브라우저를 실행하여 서버의 끝에서 끝까지의 테스트를 기록하고 실행합니다.테스트를 기록하기 위해서, 순식간에 서버를 가진 웹소켓을 만들어서 저장할 수 있어야 합니다.
Fargate가 있는 ECS는 서버를 실행하는 위험과 조작 비용을 없애지만 Kubernetes가 도입하는 복잡성이 없기 때문에 좋은 선택이다.
우리의 첫 번째 디자인은 Application Load Balanced Fargate Service 모델을 사용했지만 다음과 같은 문제에 부딪혔다.
논리 구성 요소
*.eu-west-2.browser.reflow.io
TeamId
를 베이킹합니다. 이것은cognito 사용자 정의 속성으로 클라이언트 JWT에서 모든 웹에 서명을 요청해야 합니다.LetsEncrypt SSL 인증서
CDK
우리는 CDK를 사용하여 우리의 모든 인프라를 관리한다.다음은 LetsEncrypt Lambda 함수를 관리하는 데 사용되는 구성입니다.
다음과 같은 이점을 제공합니다.
ReamplifyLambdaFunction
가 하나 있지만 이것도 포장기NodeJSFunction일 수 있다.domain
을 놓을 것입니다.workspace
(예: dev
/prod
.이것은 AWS 계정에서 이 구조의 여러 실례를 제공할 수 있도록 합니다.region/account
import { Construct } from 'constructs';
import { Duration, RemovalPolicy, StackProps, Tags } from 'aws-cdk-lib';
import { BlockPublicAccess, Bucket, BucketEncryption, ObjectOwnership } from 'aws-cdk-lib/aws-s3';
import { Topic } from 'aws-cdk-lib/aws-sns';
import { EmailSubscription } from 'aws-cdk-lib/aws-sns-subscriptions';
import { ReamplifyLambdaFunction } from './reamplifyLambdaFunction';
import { PolicyStatement } from 'aws-cdk-lib/aws-iam';
import { IHostedZone } from 'aws-cdk-lib/aws-route53';
import { Rule, Schedule } from 'aws-cdk-lib/aws-events';
import { LambdaFunction } from 'aws-cdk-lib/aws-events-targets';
interface CertbotProps {
adminNotificationEmail: string;
hostedZone: IHostedZone;
domain: string;
workspace: string;
env: {
region: string;
account: string;
};
}
export class Certbot extends Construct {
public readonly certBucket: Bucket;
constructor(scope: Construct, id: string, props: StackProps & CertbotProps) {
super(scope, id);
Tags.of(this).add('construct', 'Certbot');
const certBucket = new Bucket(this, 'bucket', {
bucketName: `certs.${props.env.region}.${props.workspace}.reflow.io`,
objectOwnership: ObjectOwnership.BUCKET_OWNER_PREFERRED,
removalPolicy: RemovalPolicy.DESTROY,
autoDeleteObjects: true,
versioned: true,
lifecycleRules: [
{
enabled: true,
abortIncompleteMultipartUploadAfter: Duration.days(1),
},
],
encryption: BucketEncryption.S3_MANAGED,
enforceSSL: true,
blockPublicAccess: BlockPublicAccess.BLOCK_ALL,
});
this.certBucket = certBucket;
const topic = new Topic(this, 'CertAdminTopic');
topic.addSubscription(new EmailSubscription(props.adminNotificationEmail));
const fn = new ReamplifyLambdaFunction(this, 'LambdaFn', {
workspace: props.workspace,
lambdaConfig: 'deploy/browserCerts.ts',
timeout: Duration.minutes(15),
environment: {
NOTIFY_EMAIL: props.adminNotificationEmail,
CERTIFICATES: JSON.stringify([
{
domains: [`*.${props.domain}`],
zoneId: props.hostedZone.hostedZoneId,
certStorageBucketName: certBucket.bucketName,
certStoragePrefix: 'browser/',
successSnsTopicArn: topic.topicArn,
failureSnsTopicArn: topic.topicArn,
},
]),
},
});
fn.addToRolePolicy(
new PolicyStatement({
actions: ['route53:ListHostedZones'],
resources: ['*'],
})
);
fn.addToRolePolicy(
new PolicyStatement({
actions: ['route53:GetChange', 'route53:ChangeResourceRecordSets'],
resources: ['arn:aws:route53:::change/*'].concat(props.hostedZone.hostedZoneArn),
})
);
fn.addToRolePolicy(
new PolicyStatement({
actions: ['ssm:GetParameter', 'ssm:PutParameter'],
resources: ['*'],
})
);
certBucket.grantWrite(fn);
topic.grantPublish(fn);
new Rule(this, 'trigger', {
schedule: Schedule.cron({ minute: '32', hour: '17', day: '3', month: '*', year: '*' }),
targets: [new LambdaFunction(fn)],
});
}
}
AWS Lambda 함수
종속성:
acme-client
: 4.2.3
acme-client
에 의존합니다.import AWS from 'aws-sdk';
import acme from 'acme-client';
const route53 = new AWS.Route53();
const s3 = new AWS.S3();
const sns = new AWS.SNS();
export function assertEnv(key: string): string {
if (process.env[key] !== undefined) {
console.log('env', key, 'resolved by process.env as', process.env[key]!);
return process.env[key]!;
}
throw new Error(`expected environment variable ${key}`);
}
export const assertEnvOrSSM = async (key: string, shouldThrow = true): Promise<string> => {
const workspace = assertEnv('workspace');
if (process.env[key] !== undefined) {
console.log('env', key, 'resolved by process.env as', process.env[key]!);
return Promise.resolve(process.env[key]!);
} else {
const SSMLocation = `/${workspace}/${key}`;
console.log('env', key, 'resolving via SSM at', SSMLocation);
const SSM = new AWS.SSM();
try {
const ssmResponse = await SSM.getParameter({
Name: SSMLocation,
}).promise();
if (!ssmResponse.Parameter || !ssmResponse.Parameter.Value) {
throw new Error(`env ${key} missing`);
}
console.log('env', key, 'resolved by SSM as', ssmResponse.Parameter.Value);
process.env[key] = ssmResponse.Parameter.Value;
return ssmResponse.Parameter.Value;
} catch (e) {
console.error(`SSM.getParameter({Name: ${SSMLocation}}):`, e);
if (shouldThrow) {
throw e;
}
return '';
}
}
};
export const writeSSM = async (key: string, value: string): Promise<void> => {
const workspace = assertEnv('workspace');
const SSMLocation = `/${workspace}/${key}`;
console.log('env', key, 'writing to SSM at', SSMLocation, 'value', value);
const SSM = new AWS.SSM();
await SSM.putParameter({
Name: SSMLocation,
Value: value,
Overwrite: true,
DataType: 'text',
Tier: 'Standard',
Type: 'String',
}).promise();
};
async function getOrCreateAccountPrivateKey() {
let accountKey = await assertEnvOrSSM('LETSENCRYPT_ACCOUNT_KEY', false);
if (accountKey) {
return accountKey;
}
console.log('Generating Account Key');
accountKey = (await acme.forge.createPrivateKey()).toString();
await writeSSM('LETSENCRYPT_ACCOUNT_KEY', accountKey);
return accountKey;
}
export const handler = async function (event) {
const maintainerEmail = assertEnv('NOTIFY_EMAIL');
const accountURL = await assertEnvOrSSM('LETSENCRYPT_ACCOUNT_URL', false);
const certificates = JSON.parse(assertEnv('CERTIFICATES'));
const accountPrivateKey = await getOrCreateAccountPrivateKey();
acme.setLogger(console.log);
const client = new acme.Client({
directoryUrl: acme.directory.letsencrypt.production,
accountKey: accountPrivateKey,
accountUrl: accountURL ? accountURL : undefined,
});
const certificateRuns = certificates.map(async (certificate) => {
const { domains, zoneId, certStorageBucketName, certStoragePrefix, successSnsTopicArn, failureSnsTopicArn } =
certificate;
try {
const [certificateKey, certificateCsr] = await acme.forge.createCsr({
commonName: domains[0],
altNames: domains.slice(1),
});
const certificate = await client.auto({
csr: certificateCsr,
email: maintainerEmail,
termsOfServiceAgreed: true,
challengeCreateFn: async (authz, challenge, keyAuthorization) => {
console.log(authz, challenge, keyAuthorization);
const dnsRecord = `_acme-challenge.${authz.identifier.value}`;
if (challenge.type !== 'dns-01') {
throw new Error('Only DNS-01 challenges are supported');
}
const changeReq = {
ChangeBatch: {
Changes: [
{
Action: 'UPSERT',
ResourceRecordSet: {
Name: dnsRecord,
ResourceRecords: [
{
Value: '"' + keyAuthorization + '"',
},
],
TTL: 60,
Type: 'TXT',
},
},
],
},
HostedZoneId: zoneId,
};
console.log('Sending create request', JSON.stringify(changeReq));
const response = await route53.changeResourceRecordSets(changeReq).promise();
const changeId = response.ChangeInfo.Id;
console.log(`Create request sent for ${dnsRecord} (Change id ${changeId}); waiting for it to complete`);
const waitRequest = route53.waitFor('resourceRecordSetsChanged', { Id: changeId });
const waitResponse = await waitRequest.promise();
console.log(
`Create request complete for ${dnsRecord}: (Change id ${waitResponse.ChangeInfo.Id}) ${waitResponse.ChangeInfo.Status}`
);
},
challengeRemoveFn: async (authz, challenge, keyAuthorization) => {
const dnsRecord = `_acme-challenge.${authz.identifier.value}`;
const deleteReq = {
ChangeBatch: {
Changes: [
{
Action: 'DELETE',
ResourceRecordSet: {
Name: dnsRecord,
ResourceRecords: [
{
Value: '"' + keyAuthorization + '"',
},
],
TTL: 60,
Type: 'TXT',
},
},
],
},
HostedZoneId: zoneId,
};
console.log('Sending delete request', JSON.stringify(deleteReq));
const response = await route53.changeResourceRecordSets(deleteReq).promise();
const changeId = response.ChangeInfo.Id;
console.log(`Delete request sent for ${dnsRecord} (Change id ${changeId}); waiting for it to complete`);
const waitRequest = route53.waitFor('resourceRecordSetsChanged', { Id: changeId });
const waitResponse = await waitRequest.promise();
console.log(
`Delete request complete for ${dnsRecord}: (Change id ${waitResponse.ChangeInfo.Id}) ${waitResponse.ChangeInfo.Status}`
);
},
challengePriority: ['dns-01'],
});
// Write private key & certificate to S3
const certKeyWritingPromise = s3
.putObject({
Body: certificateKey.toString(),
Bucket: certStorageBucketName,
Key: certStoragePrefix + 'key.pem',
ServerSideEncryption: 'AES256',
})
.promise();
const certChainWritingPromise = s3
.putObject({
Body: certificate,
Bucket: certStorageBucketName,
Key: certStoragePrefix + 'cert.pem',
})
.promise();
await Promise.all([certKeyWritingPromise, certChainWritingPromise]);
console.log('Completed with certificate for ', domains);
// after client.auto, an account should be available
if (!accountURL) {
await writeSSM('LETSENCRYPT_ACCOUNT_URL', client.getAccountUrl());
}
if (successSnsTopicArn) {
await sns
.publish({
TopicArn: successSnsTopicArn,
Message: `Certificate for ${JSON.stringify(domains)} issued`,
Subject: 'Certificate Issue Success',
})
.promise();
}
} catch (err) {
console.log('Error ', err);
if (failureSnsTopicArn) {
await sns
.publish({
TopicArn: failureSnsTopicArn,
Message: `Certificate for ${JSON.stringify(domains)} issue failure\n${err}`,
Subject: 'Certificate Issue Failure',
})
.promise();
}
throw err;
}
});
await Promise.all(certificateRuns);
};
자동 DNS 로깅
CDK
이것은 다음과 같습니다.
clusterArn
클러스터serviceDiscoveryTLD
(우리의 예에서는browser.${props.env.region}.reflow.io
)를 DNS 기록import { Rule } from 'aws-cdk-lib/aws-events';
import { LambdaFunction } from 'aws-cdk-lib/aws-events-targets';
import { PolicyStatement } from 'aws-cdk-lib/aws-iam';
// ...
const eventRule = new Rule(this, 'ECSChangeRule', {
eventPattern: {
source: ['aws.ecs'],
detailType: ['ECS Task State Change'],
detail: {
clusterArn: [cluster.clusterArn],
},
},
});
const ecsChangeFn = new ReamplifyLambdaFunction(this, 'ECSStreamLambda', {
...props,
lambdaConfig: 'stream/ecsChangeStream.ts',
unreservedConcurrency: true,
memorySize: 128,
environment: {
DOMAIN_PREFIX: props.serviceDiscoveryTLD,
HOSTED_ZONE_ID: props.hostedZone.hostedZoneId,
},
});
eventRule.addTarget(new LambdaFunction(ecsChangeFn));
ecsChangeFn.addToRolePolicy(
new PolicyStatement({
actions: ['route53:GetChange', 'route53:ChangeResourceRecordSets', 'route53:ListResourceRecordSets'],
resources: ['arn:aws:route53:::change/*'].concat(props.hostedZone.hostedZoneArn),
})
);
ecsChangeFn.addToRolePolicy(
new PolicyStatement({
actions: ['ec2:DescribeNetworkInterfaces'],
resources: ['*'],
})
);
AWS Lambda
이 기능은 다음과 같습니다.
RUNNING
및 필요한 경우RUNNING
:A
레코드 삽입${taskId}.${DOMAIN_PREFIX}
기록을 삭제합니다.import type { EventBridgeHandler } from 'aws-lambda';
import AWS from 'aws-sdk';
import { Task } from 'aws-sdk/clients/ecs';
export function assertEnv(key: string): string {
if (process.env[key] !== undefined) {
console.log('env', key, 'resolved by process.env as', process.env[key]!);
return process.env[key]!;
}
throw new Error(`expected environment variable ${key}`);
}
const ec2 = new AWS.EC2();
const route53 = new AWS.Route53();
const DOMAIN_PREFIX = assertEnv('DOMAIN_PREFIX');
const HOSTED_ZONE_ID = assertEnv('HOSTED_ZONE_ID');
export const handler: EventBridgeHandler<string, Task, unknown> = async (event) => {
console.log('event', JSON.stringify(event));
const task = event.detail;
const clusterArn = task.clusterArn;
const lastStatus = task.lastStatus;
const desiredStatus = task.desiredStatus;
if (!clusterArn) {
return;
}
if (!lastStatus) {
return;
}
if (!desiredStatus) {
return;
}
const taskArn = task.taskArn;
if (!taskArn) {
return;
}
const taskId = taskArn.split('/').pop();
if (!taskId) {
return;
}
const clusterName = clusterArn.split(':cluster/')[1];
if (!clusterName) {
return;
}
const containerDomain = `${taskId}.${DOMAIN_PREFIX}`;
if (lastStatus === 'RUNNING' && desiredStatus === 'RUNNING') {
const eniId = getEniId(task);
if (!eniId) {
return;
}
const taskPublicIp = await fetchEniPublicIp(eniId);
if (!taskPublicIp) {
return;
}
const recordSet = createRecordSet(containerDomain, taskPublicIp);
await updateDnsRecord(clusterName, HOSTED_ZONE_ID, recordSet);
console.log(`DNS record update finished for ${taskId} (${taskPublicIp})`);
} else {
const recordSet = await route53
.listResourceRecordSets({
HostedZoneId: HOSTED_ZONE_ID,
StartRecordName: containerDomain,
StartRecordType: 'A',
})
.promise();
console.log('listRecordSets', JSON.stringify(recordSet));
const found = recordSet.ResourceRecordSets.find((record) => record.Name === containerDomain + '.');
if (found && found.ResourceRecords?.[0].Value) {
await route53
.changeResourceRecordSets({
HostedZoneId: HOSTED_ZONE_ID,
ChangeBatch: {
Changes: [
{
Action: 'DELETE',
ResourceRecordSet: {
Name: containerDomain,
Type: 'A',
ResourceRecords: [
{
Value: found.ResourceRecords[0].Value,
},
],
TTL: found.TTL,
},
},
],
},
})
.promise();
}
}
};
function getEniId(task): string | undefined {
const eniAttachment = task.attachments.find(function (attachment) {
return attachment.type === 'eni';
});
if (!eniAttachment) {
return undefined;
}
const networkInterfaceIdDetail = eniAttachment.details.find((detail) => detail.name === 'networkInterfaceId');
if (!networkInterfaceIdDetail) {
return undefined;
}
return networkInterfaceIdDetail.value;
}
async function fetchEniPublicIp(eniId): Promise<string | undefined> {
const data = await ec2
.describeNetworkInterfaces({
NetworkInterfaceIds: [eniId],
})
.promise();
console.log(data);
return data.NetworkInterfaces?.[0].PrivateIpAddresses?.[0].Association?.PublicIp;
}
function createRecordSet(domain, publicIp) {
return {
Action: 'UPSERT',
ResourceRecordSet: {
Name: domain,
Type: 'A',
TTL: 60,
ResourceRecords: [
{
Value: publicIp,
},
],
},
};
}
async function updateDnsRecord(clusterName, hostedZoneId, changeRecordSet) {
let param = {
ChangeBatch: {
Comment: `Auto generated Record for ECS Fargate cluster ${clusterName}`,
Changes: [changeRecordSet],
},
HostedZoneId: hostedZoneId,
};
await route53.changeResourceRecordSets(param).promise();
}
프로덕션에서 실행
이것은 이미 생산에 들어간 지 두 달이 되었다. 비록 그것은 완벽하지는 않지만, 우리에게는 효과가 매우 좋다.
우리가 걱정할 필요가 없는 일:
Reference
이 문제에 관하여(LetsEncrypt, CDK 및 AWS Lambda를 사용하여 ECS Fargate 컨테이너에 직접 액세스(나쁜 생각일 수 있음)), 우리는 이곳에서 더 많은 자료를 발견하고 링크를 클릭하여 보았다 https://dev.to/aws-builders/get-direct-traffic-to-ecs-fargate-containers-with-letsencrypt-cdk-and-aws-lambda-probably-a-bad-idea-3la7텍스트를 자유롭게 공유하거나 복사할 수 있습니다.하지만 이 문서의 URL은 참조 URL로 남겨 두십시오.
우수한 개발자 콘텐츠 발견에 전념 (Collection and Share based on the CC Protocol.)