EC2의 취약성 진단을 Inspector로 자동화해 보았다

이 기사는 삼작 #1 Advent Calendar 2020의 20 일째 기사입니다.

Amazon Inspector란?



Inspector에 대한 자세한 내용은 AWS 공식 문서를 참조하세요.
htps : // 아 ws. 아마존. 이 m/jp/인 spec와 r/

실현하고 싶은 것



정기적으로 자동으로 EC2 인스턴스를 시작하고 Inspector에서 취약점을 확인하고 Slack에 알립니다. 기동중의 인스턴스를 접직 체크는 하고 싶지 않다.
위의 요건을 바탕으로 다음 구성으로 정기적으로 취약성 검사를 실시하기로 했습니다.

  • Amazon EventBridge에서 Lambda 실행
  • Lambda에서 취약점을 확인하려는 EC2 인스턴스의 AMI에서 인스턴스 시작
  • 시작된 인스턴스를 Amazon Inspector에서 취약점 검사
  • Inspector 진단이 완료되면 SNS를 통해 진단 결과를 얻고 Slack에 알리는 Lambda를 실행합니다.

    AWS 리소스 생성은 모두 Terraform에서 코드를 관리할 수 있습니다.

    Inspector 설정



    inspector.tf
    ######################################
    # 評価ターゲット作成
    ######################################
    resource "aws_inspector_resource_group" "resource" {
      tags = {
        Inspector = "true"
      }
    }
    
    resource "aws_inspector_assessment_target" "target" {
      name               = "test-target"
      resource_group_arn = aws_inspector_resource_group.resource.arn
    }
    
    ######################################
    # Rule Package 取得
    ######################################
    data "aws_inspector_rules_packages" "rules" {}
    
    ######################################
    # 評価テンプレート作成
    ######################################
    resource "aws_inspector_assessment_template" "template" {
      name               = "test-template"
      target_arn         = aws_inspector_assessment_target.target.arn
      duration           = 3600
      rules_package_arns = data.aws_inspector_rules_packages.rules.arns
    }
    

    · 평가 타겟 만들기
    Tag:Inspector:true가 설정된 EC2 인스턴스를 대상으로 합니다.

    · 평가 Rule Package 취득
    아래의 모든 Rule Package에서 평가 템플릿을 얻으십시오.
    Common Vulnerabilities and Exposures: 공통 취약성 식별자
    Security Best Practices: Amazon Inspector 보안 모범 사례
    Center for Internet Security (CIS) Benchmarks: Center for Internet Security (CIS) 벤치마크
    Network Reachability: 네트워크 도달 가능성

    · 평가 템플릿 작성
    만든 평가 타겟팅 설정
    실행 시간을 1시간으로 설정
    획득한 평가 Rule Package 설정

    Lambda 설정



    Lambda 함수는 다음 두 가지 함수를 만듭니다.
    · AMI를 기동하여 Inspector 평가를 실행하는 Lambda
    · Inspector 평가 결과를 Slack에 알리는 Lambda

    AMI를 시작하고 Inspector 평가를 수행하는 Lambda
    function_inspector_run.tf
    ######################################
    # Lambda IAM Role作成
    ######################################
    resource "aws_iam_role" "lambda_role" {
      name = "lambda-inspector-role"
      assume_role_policy = <<EOF
    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": {
            "Service": "lambda.amazonaws.com"
          },
          "Action": "sts:AssumeRole"
        }
      ]
    }
    EOF
    }
    
    ######################################
    # Lambda IAM Policy作成
    ######################################
    resource "aws_iam_policy" "lambda_policy" {
      name        = "lamda-inspector-policy"
      description = "LambdaFunction function_inspector_run/function_inspector_report Use Policy"
      policy = <<EOF
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "VisualEditor0",
                "Effect": "Allow",
                "Action": [
                    "inspector:GetAssessmentReport",
                    "inspector:ListAssessmentTemplates",
                    "inspector:ListAssessmentTargets",
                    "inspector:ListAssessmentRuns",
                    "inspector:StartAssessmentRun",
                    "inspector:PreviewAgents"
                ],
                "Resource": "*"
            },
            {
                "Sid": "VisualEditor1",
                "Effect": "Allow",
                "Action": [
                    "logs:CreateLogGroup",
                    "logs:CreateLogStream",
                    "logs:PutLogEvents"
                ],
                "Resource": "*"
            }
        ]
    }
    EOF
    }
    
    ######################################
    # RoleにPolicyをアタッチ
    ######################################
    resource "aws_iam_role_policy_attachment" "lambda_role_policy_attach" {
      role       = aws_iam_role.lambda_role.name
      policy_arn = aws_iam_policy.lambda_policy.arn
    }
    
    ######################################
    # Lambda関数の作成
    ######################################
    resource "aws_lambda_function" "inspector_run" {
      filename         = "./function_inspector_run.zip"
      function_name    = "function_inspector_run"
      role             = aws_iam_role.lambda_role.arn
      handler          = "lambda_function.lambda_handler"
      source_code_hash = filebase64sha256("./function_inspector_run.zip")
      runtime          = "python3.8"
      timeout          = 300
      environment {
        variables = {
          TZ               = "Asia/Tokyo"
          SECURITYGROUP_ID = "xxxxxx"
          SUBNET_ID        = "xxxxxx"
        }
      }
    }
    

    function_inspector_run.py
    import boto3
    import os
    from operator import itemgetter
    import logging
    import datetime
    import time
    
    logger = logging.getLogger()
    logger.setLevel(logging.INFO)
    
    def lambda_handler(event, context):
        create_ec2_instance()
        start_inspector_assessment_run()
    
    def create_ec2_instance():
        """
        Inspectorで分析するため最新のAMIを取得しAMIからEC2Instanceを起動します
        起動設定でawsagentをinstallします
        """
        # web AMI一覧取得
        ec2 = boto3.client('ec2')
        # https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Client.describe_images
        response = ec2.describe_images(
            Filters=[
                {
                    "Name": "tag:Name",
                    "Values": [
                        "test-web*",
                    ]
                }
            ],
            Owners=[
                "self"
            ]
        )
        # 最新のAMIを取得
        image_details = sorted(response['Images'], key=itemgetter('CreationDate'), reverse=True)
        image_id = image_details[0]['ImageId']
        user_data = """#!/bin/sh
        sudo su -
        wget https://inspector-agent.amazonaws.com/linux/latest/install
        bash install -u false
        /etc/init.d/awsagent restart
        """
        # Instance作成/起動
        # https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Client.run_instances
        response = ec2.run_instances(
            ImageId=image_id,
            InstanceType="t2.micro",
            MinCount=1,
            MaxCount=1,
            KeyName="test_key_pair",
            SecurityGroupIds=[
                os.environ['SECURITYGROUP_ID']
            ],
            SubnetId=os.environ['SUBNET_ID'],
            UserData=user_data,
            TagSpecifications=[
                {
                    'ResourceType': 'instance',
                    'Tags': [
                        {
                            'Key': 'Inspector',
                            'Value': 'true'
                        },
                        {
                            'Key': 'Name',
                            'Value': 'inspector-target'
                        }
                    ]
                }
            ]
        )
        # EC2Instanceがstatus:runningになるまで待ち
        instance_id = response.get('Instances')[0]['InstanceId']
        while 1:
            time.sleep(30)
            instances = ec2.describe_instances(
                InstanceIds=[
                    instance_id
                ]
            )
            logger.info(instances.get('Reservations')[0].get('Instances')[0].get('State').get('Name'))
            if instances.get('Reservations')[0].get('Instances')[0].get('State').get('Name') == 'running':
                break
    
    def start_inspector_assessment_run():
        """
        Inspector評価実行を行います
        """
        # 評価テンプレート一覧取得
        inspector = boto3.client('inspector')
        # 評価ターゲットのEC2Instance awsagentのステータスがHEALTHYになるまで待機
        assessment_target = inspector.list_assessment_targets(
            filter={
                'assessmentTargetNamePattern': os.environ['ENV'] + '-target'
            }
        )
        while 1:
            time.sleep(30)
            awsagents_status = inspector.preview_agents(previewAgentsArn=assessment_target.get('assessmentTargetArns')[0])
            logger.info(awsagents_status.get('agentPreviews')[0])
            if awsagents_status.get('agentPreviews')[0].get('agentHealth') == 'HEALTHY':
                break
        """
        list_assessment_templates Response Syntax
        {
            'assessmentTemplateArns': [
                'string',
            ],
            'nextToken': 'string'
        }
        https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/inspector.html#Inspector.Client.list_assessment_templates
        """
        assessment_templates = inspector.list_assessment_templates(
            filter={
                'namePattern': 'test-template'
            }
        )
        # 評価テンプレートARN一覧取得
        if not assessment_templates.get('assessmentTemplateArns'):
            return
        logger.info(assessment_templates.get('assessmentTemplateArns'))
        # 評価の実行
        for template_arn in assessment_templates.get('assessmentTemplateArns'):
            inspector.start_assessment_run(
                assessmentTemplateArn=template_arn,
                assessmentRunName='RunAssessment_' + 'test-template' + datetime.date.today().strftime("%Y-%m-%d")
            )
    

    Inspector 평가 결과를 Slack에 알리는 Lambda
    function_inspector_report.tf
    resource "aws_lambda_function" "inspector_report" {
      filename         = "./function_inspector_report.zip"
      function_name    = "function_inspector_report"
      role             = aws_iam_role.lambda_role.arn
      handler          = "lambda_function.lambda_handler"
      source_code_hash = filebase64sha256("./function_inspector_report.zip")
      runtime          = "python3.8"
      timeout          = 300
      environment {
        variables = {
          TZ       = "Asia/Tokyo"
          SLACK_HOOK_URL = "https://xxxxxxxxxxxxx"
        }
      }
    }
    
    resource "aws_lambda_permission" "inspector_report_permission" {
      action        = "lambda:InvokeFunction"
      function_name = aws_lambda_function.inspector_report.function_name
      principal     = "sns.amazonaws.com"
    }
    
    

    function_inspector_report.py
    import json
    import boto3
    import os
    import logging
    import datetime
    from urllib.request import Request, urlopen
    from urllib.error import URLError, HTTPError
    import time
    
    logger = logging.getLogger()
    logger.setLevel(logging.INFO)
    
    def lambda_handler(event, context):
    
        assessment_run_arn = get_assessment_run_arn(event)
        if not assessment_run_arn:
            logger.info('AssessmentTarget Not Found')
            logger.info(event)
            return
    
        send_inspector_assessment_report(assessment_run_arn)
        terminate_ec2_instance()
    
    def get_assessment_run_arn(event):
        """
          評価実行のARN取得処理
          SNS, Lambdaテスト実行の判定を行いInspector評価実行のARNを返却します
          Args:
              event : Lambda実行Parameter
          Return:
              assessment_run_arn: Inspector 評価実行ARN
        """
        inspector = boto3.client('inspector')
        # SNSからのLambda実行
        if event.get('Records'):
            logger.info('SNS execute')
            message = event['Records'][0]['Sns']['Message']
            message = json.loads(message)
            logger.info(message)
            return message.get('run')
        # Lambda テスト実行パラメータ有
        elif event.get('target_date'):
            logger.info('Lambda execute Parameter ON target_date =' + event.get('target_date'))
            """
            list_assessment_runs Response Syntax
            {
                'assessmentRunArns': [
                    'string',
                ],
                'nextToken': 'string'
            }
            https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/inspector.html#Inspector.Client.list_assessment_runs
            """
            assessment_runs = inspector.list_assessment_runs(
                filter={
                    'namePattern':'RunAssessment_' + 'test-template' + event.get('target_date')
                }
            )
            return assessment_runs.get('assessmentRunArns')[0]
        # Lambda テスト実行パラメータ無
        else:
            logger.info('Lambda execute Parameter OFF')
            assessment_runs = inspector.list_assessment_runs(
                filter={
                    'namePattern':'RunAssessment_' + 'test-template' + datetime.date.today().strftime("%Y-%m-%d")
                }
            )
            return assessment_runs.get('assessmentRunArns')[0]
    
    def send_inspector_assessment_report(assessment_run_arn):
        """
        Inspector評価実行結果をSlackに送信します
        """
        inspector = boto3.client('inspector')
        report = {}
        """
        get_assessment_report Response Syntax
        {
            'status': 'WORK_IN_PROGRESS'|'FAILED'|'COMPLETED',
            'url': 'string'
        }
        https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/inspector.html#Inspector.Client.get_assessment_report
        """
        # すぐ結果を取得しても取得できない時があるのでStatusがCOMPLETEDになるまで待機
        while 1:
            time.sleep(30);
            # HTML形式でReport出力
            assessment_report_html = inspector.get_assessment_report(
                assessmentRunArn=assessment_run_arn,
                reportFileFormat='HTML',
                reportType='FULL'
            )
            if assessment_report_html.get('status') == 'COMPLETED':
                break
    
        # すぐ結果を取得しても取得できない時があるのでStatusがCOMPLETEDになるまで待機
        while 1:
            time.sleep(30);
            # PDF形式でReport出力
            assessment_report_pdf = inspector.get_assessment_report(
                assessmentRunArn=assessment_run_arn,
                reportFileFormat='PDF',
                reportType='FULL'
            )
            logger.info(assessment_report_pdf.get('status'))
            if assessment_report_pdf.get('status') == 'COMPLETED':
                break
    
        report['arn'] = assessment_run_arn
        report['html'] = assessment_report_html.get('url')
        report['pdf'] = assessment_report_pdf.get('url')
    
        channel = '#' + 'inspector-report'
        message = "*Inspector 評価実行結果*\n*Arn*:```{}```\n\n*HTML結果*:```{}```\n\n*PDF結果*:```{}```\n\n※ページの有効期限が900sで切れます。有効期限が切れた時はLambda関数:function_inspector_reportでテスト実行またはawscliから実行を行なってください\n```テストイベントのパラメータ:{}\naws inspector get-assessment-report --assessment-run-arn {} --report-file-format PDF --report-type FULL```"
        slack_message = {
            'channel': channel,
            'text': message.format(
                report.get('arn'),
                report.get('html'),
                report.get('pdf'),
                '{"target_date": YYYYMMDD}',
                report.get('arn')
            )
        }
        HOOK_URL = os.environ['SLACK_HOOK_URL']
    
        req = Request(HOOK_URL, json.dumps(slack_message).encode('UTF-8'))
        try:
            response = urlopen(req)
            response.read()
        except HTTPError as e:
            logger.error("Request failed: %d %s", e.code, e.reason)
        except URLError as e:
            logger.error("Server connection failed: %s", e.reason)
    
    def terminate_ec2_instance():
        """
        Inspectorで分析したEC2InstanceをTerminateします
        """
        ec2 = boto3.client('ec2')
        # tag:Inspector:trueのEC2Instance一覧取得
        # https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Client.describe_instances
        instances = ec2.describe_instances(Filters=[{
            'Name': 'tag:Inspector',
            'Values': ['true']
        }])
    
        if not instances:
            return
    
        # InstanceIDの配列生成
        instance_ids = []
        for reservation in instances.get('Reservations'):
            for instance in reservation.get('Instances'):
                logger.info(instance.get('InstanceId'))
                instance_ids.append(instance.get('InstanceId'))
    
        if not instance_ids:
            return
    
        # 対象EC2InstanceのTerminate
        # https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#EC2.Client.terminate_instances
        ec2.terminate_instances(
            InstanceIds=instance_ids
        )
    

    CloudWatchEvents 설정



    마지막으로 Amazon EventBridge에서 Inspector 실행 Lambda를 예약하고 완료했습니다.

    실행해 본 결과



    Slack에 다음 메시지가 표시되었음을 확인했습니다! !
    Inspector 評価実行結果
    Arn:
    arn:aws:inspector:xxxx:xxxx:target/xxxx/template/xxxx/run/xxxx
    HTML結果:
    https://inspector-html-report
    PDF結果:
    https://inspector-pdf-report
    
    ※ページの有効期限が900sで切れます。有効期限が切れた時はLambda関数:function_inspector_reportでテスト実行またはawscliから実行を行なってください
    テストイベントのパラメータ:{"target_date": YYYYMMDD}
    aws inspector get-assessment-report --assessment-run-arn xxx --report-file-format PDF or HTML --report-type FULL
    
  • 좋은 웹페이지 즐겨찾기