아마존 검사기를 사용하기 위해awsagent를 설치하는 환경이 있습니다.처음에는 문제가 없었던awsagent가 몇 시간 뒤 오류가 생겨 메시지를 남겼다.

요점

IMDSv2의 강제 설정과 awsagent의 시작 순서가 중요합니다.IMDSv2強制設定 ⇒ awsagentの起動

이벤트

설치 후 몇 시간 동안 정상적으로 작동할 수 있지만 다음 날 등 Inspector 화면을 보면 Healthy가 아닌 UNKNOWN 상태입니다.

나는 방금 호스트에 로그인해서 로그를 보았다.시작 후 몇 시간 후에 오류가 발견됩니다.

$ sudo journalctl --full --no-pager  -u awsagent.service
#省略--
Jan 03 23:44:50 ip-xxx awsagent[4603]: Region : 'ap-northeast-1'
Jan 03 23:44:50 ip-xxx awsagent[4603]: [  OK  ]
Jan 03 23:44:50 ip-xxx systemd[1]: Started LSB: Amazon Inspector service.
Jan 03 23:47:57 ip-xxx awsagent[4669]: Core/MetaDataClient.cpp:88:HTTP Request on URL http://169.254.169.254/latest/api/token to EC2 metadata client service returned nullptr.
Jan 03 23:47:57 ip-xxx awsagent[4669]: Core/MetaDataClient.cpp:106:HTTP Request on URL http://169.254.169.254/latest/meta-data/placement/availability-zone to EC2 metadata client service failed with HttpResponseError: 401 on retry 0.
Jan 03 23:47:57 ip-xxx awsagent[4669]: Publishers/Publisher.cpp:382:Failed to retrieve availability zone: -97
Jan 04 00:48:06 ip-xxx awsagent[4669]: Publishers/ArsenalPublisher.cpp:1588:PublishEvents operation not permitted failure : Publishing data to service with no subscription.
Jan 04 02:05:25 ip-xxx awsagent[4669]: Core/MetaDataClient.cpp:88:HTTP Request on URL http://169.254.169.254/latest/meta-data/placement/availability-zone to EC2 metadata client service returned nullptr.
Jan 04 02:05:25 ip-xxx awsagent[4669]: Publishers/Publisher.cpp:382:Failed to retrieve availability zone: -1
Jan 04 02:58:07 ip-xxx awsagent[4669]: Core/MetaDataClient.cpp:106:HTTP Request on URL http://169.254.169.254/latest/meta-data/placement/availability-zone to EC2 metadata client service failed with HttpResponseError: 401 on retry 0.
Jan 04 06:19:13 ip-xxx awsagent[4669]: Publishers/ArsenalPublisher.cpp:742:RequestConfig failure : Unable to parse ExceptionName: ExpiredTokenException Message: The security token included in the request is expired
Jan 04 06:19:13 ip-xxx awsagent[4669]: Service/MainInspectorThread.cpp:79:Config retrieval failed : -97
Jan 04 06:21:13 ip-xxx awsagent[4669]: Publishers/ArsenalPublisher.cpp:557:UpdateHealth failure : Unable to parse ExceptionName: ExpiredTokenException Message: The security token included in the request is expired Msg: {"t":1609741273921,"proxy":0,"o":"Amazon Linux release 2 (Karoo)","k":"4.14.209-160.335.amzn2.x86_64","r":"Unrecognized failure : Unable to parse ExceptionName: ExpiredTokenException Message: The security token included in the request is expired","s":17,"d":0,"l":51,"m":0}
Jan 04 06:21:13 ip-xxx awsagent[4669]: Service/MainInspectorThread.cpp:96:Regular health update failed : 0
Jan 04 07:19:20 ip-xxx awsagent[4669]: Publishers/ArsenalPublisher.cpp:742:RequestConfig failure : Unable to parse ExceptionName: ExpiredTokenException Message: The security token included in the request is expired
Jan 04 07:19:20 ip-xxx awsagent[4669]: Service/MainInspectorThread.cpp:79:Config retrieval failed : -97
Jan 04 07:21:20 ip-xxx awsagent[4669]: Publishers/ArsenalPublisher.cpp:557:UpdateHealth failure : Unable to parse ExceptionName: ExpiredTokenException Message: The security token included in the request is expired Msg: {"t":1609744880720,"proxy":0,"o":"Amazon Linux release 2 (Karoo)","k":"4.14.209-160.335.amzn2.x86_64","r":"Unrecognized failure : Unable to parse ExceptionName: ExpiredTokenException Message: The security token included in the request is expired","s":17,"d":0,"l":132,"m":0}
Jan 04 07:21:20 ip-xxx awsagent[4669]: Service/MainInspectorThread.cpp:96:Regular health update failed : 0

신경 쓰이는 것은 다음과 같은 정보다.The security token included in the request is expired
IMDSv2를 사용하고 푹 빠진 후에awsagent가 IMDSv2를 지원하지 않는다고 의심하여 지원을 문의했습니다.

해결됨

몇 차례의 교환 조사 결과는 개요에 기재된 바와 같다.
에이전트 awsagent가 IMDSv1과 IMDSv2를 지원하는지 확인하지만, 서비스가 시작될 때 버전을 확인하는 것 같습니다.
처음에 시작 템플릿의 사용자 데이터에서 다음과 같이 설정합니다.이렇게 하면 IMDSv2로 판별되지 않고 영패의 교환이 순조롭지 못하며 세션 영패가 만료되면서 오류가 발생한다.

curl -o /tmp/install https://inspector-agent.amazonaws.com/linux/latest/install
/bin/bash /tmp/install

aws ec2 modify-instance-metadata-options \
  --region ${REGION} \
  --instance-id ${INSTANCE_ID} \
  --http-tokens required

따라서 다음과 같이 시작 템플릿 내의 처리 순서를 바꾸어 회의장을 진행했다.

aws ec2 modify-instance-metadata-options \
  --region ${REGION} \
  --instance-id ${INSTANCE_ID} \
  --http-tokens required

# IMDSv2強制後にインストール
curl -o /tmp/install https://inspector-agent.amazonaws.com/linux/latest/install
/bin/bash /tmp/install

# またはリスタートするなど
/etc/init.d/awsagent restart