개요

CloudFormation을 사용하여 Kubernetes 설정 도구kube-aws로 AWS에 Kubernetes 환경 구축

특징으로서 다음과 같은 기능이 있다

ELB integration for Kubernetes Services allows for traffic ingress to selected microservices

Worker machines are deployed in an Auto Scaling group for effortless scaling

Full TLS is set up between Kubernetes components and users interacting with kubectl

참고
htps : // 이것 오 s. 이 m / 쿠베 r 네테 s / 드 cs / ㅁ st / 쿠베 r 네 테 s - 음 - 아 ws. HTML

환경

2016/01/14 시점에서 구축되는 내용은 이하와 같다

CoreOS-alpha-891.0.0

Docker version 1.9.1, build 4419fdb-dirty

kubernetes v1.1.2

VisualOps에서 VPC 가져 오기 구성도


master node 1대, worker node용 AutoScalingGroup가 작성된다

절차

kube-aws를 적절한 위치에 다운로드하고 압축을 풉니 다.

$ wget https://github.com/coreos/coreos-kubernetes/releases/download/v0.3.0/kube-aws-linux-amd64.tar.gz
$ tar zxvf kube-aws-linux-amd64.tar.gz

kube-aws 사용법

$ ./kube-aws --help
Manage Kubernetes clusters on AWS

Usage:
  kube-aws [command]

Available Commands:
  destroy     Destroy an existing Kubernetes cluster
  render      Render a CloudFormation template
  status      Describe an existing Kubernetes cluster
  up          Create a new Kubernetes cluster
  version     Print version information and exit
  help        Help about any command

Flags:
      --aws-debug[=false]: Log debug information from aws-sdk-go library
      --config="cluster.yaml": Location of kube-aws cluster config file


Use "kube-aws [command] --help" for more information about a command.

AWS Credentials 설정

$ export AWS_ACCESS_KEY_ID="AKXXXXXXXXXXXXXXXXXX"
$ export AWS_SECRET_ACCESS_KEY="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"

샘플 kubenetes 클러스터 구성 파일 다운로드

$ curl --silent --location https://raw.githubusercontent.com/coreos/coreos-kubernetes/master/multi-node/aws/cluster.yaml.example > cluster.yaml

클러스터 설정

$ vi cluster.yaml

cluster.yaml

# Unique name of Kubernetes cluster. In order to deploy
# more than one cluster into the same AWS account, this
# name must not conflict with an existing cluster.
clusterName: "kubernetes"

# Name of the SSH keypair already loaded into the AWS
# account being used to deploy this cluster.
keyName: "｛AWSに登録した Key pair name｝"

# Region to provision Kubernetes cluster
region: "ap-northeast-1"

# Availability Zone to provision Kubernetes cluster
availabilityZone: "ap-northeast-1a"

# DNS name routable to the Kubernetes controller nodes
# from worker nodes and external clients. The deployer
# is responsible for making this name routable
externalDNSName: "｛Kubernetes API にアクセスする際のドメイン名｝"

# Instance type for controller node
controllerInstanceType: "t2.micro"

# Disk size (GiB) for controller node
controllerRootVolumeSize: 10

# Number of worker nodes to create
workerCount: 3

# Instance type for worker nodes
workerInstanceType: "t2.micro"

# Disk size (GiB) for worker nodes
workerRootVolumeSize: 10

# Location of kube-aws artifacts used to deploy a new
# Kubernetes cluster. The necessary artifacts are already
# available in a public S3 bucket matching the version
# of the kube-aws tool. This parameter is typically
# overwritten only for development purposes.
#artifactURL: https://coreos-kubernetes.s3.amazonaws.com/<VERSION>

# CIDR for Kubernetes VPC
vpcCIDR: "10.0.0.0/16"

# CIDR for Kubernetes subnet
instanceCIDR: "10.0.0.0/24"

# IP Address for controller in Kubernetes subnet
controllerIP: 10.0.0.50

# CIDR for all service IP addresses
serviceCIDR: "10.3.0.0/24"

# CIDR for all pod IP addresses
podCIDR: "10.2.0.0/16"

# IP address of Kubernetes controller service (must be contained by serviceCIDR)
kubernetesServiceIP: "10.3.0.1"

# IP address of Kubernetes dns service (must be contained by serviceCIDR)
dnsServiceIP: "10.3.0.10"

4대로 클러스터 구축
- master :1대
- worker node:3대 (AutoScalingGroup)

배포

$ ./kube-aws up
Initialized TLS infrastructure
Wrote kubeconfig to /home/***/kube-aws/clusters/kubernetes/kubeconfig
Waiting for cluster creation...
Successfully created cluster

Cluster Name:   kubernetes
Controller IP:  xxx.xxx.xxx.xxx(EIP が表示される)

kube-aws 에 의해 작성되는 kubernetes 인증용의 열쇠와 증명서는 90일간 밖에 사용할 수 없기 때문에 주의.

PRODUCTION NOTE: the TLS keys and certificates generated by kube-aws should not be used to deploy a production Kubernetes cluster. Each component certificate is only valid for 90 days, while the CA is valid for 365 days. If deploying aclustion consider establishing PKI independently of this tool first.

도메인에서 액세스할 수 있도록 허용

$ sudo vi /etc/hosts

다음을 추가

/etc/hosts

｛Controller IP｝    ｛cluster.yaml に設定した externalDNSName｝

kubectl은 작성된 clusters/{clusterName}/kubeconfig를 사용하여 실행합니다.

$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig cluster-info
Kubernetes master is running at https://｛externalDNSName｝
KubeDNS is running at https://｛externalDNSName｝/api/v1/proxy/namespaces/kube-system/services/kube-dns

node 확인

$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig get nodes
NAME                                            LABELS                                                                 STATUS    AGE
ip-10-0-0-171.ap-northeast-1.compute.internal   kubernetes.io/hostname=ip-10-0-0-171.ap-northeast-1.compute.internal   Ready     7m
ip-10-0-0-172.ap-northeast-1.compute.internal   kubernetes.io/hostname=ip-10-0-0-172.ap-northeast-1.compute.internal   Ready     7m
ip-10-0-0-173.ap-northeast-1.compute.internal   kubernetes.io/hostname=ip-10-0-0-173.ap-northeast-1.compute.internal   Ready     7m

다음 명령은 CloudFormation template를 표시합니다.

$ ./kube-aws render

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Conditions": {
        "EmptyAvailabilityZone": {
            "Fn::Equals": [
                {
                    "Ref": "AvailabilityZone"
                },
                ""
            ]
        }
    },
    "Description": "kube-aws Kubernetes cluster",
    "Mappings": {
        "RegionMap": {
            "ap-northeast-1": {

...snip...

Kubernetes로 Wordpress를 구축해 봅니다.

매니페스트는 example을 기반으로 만듭니다.

mysql 데이터베이스 용 EBS Volume 작성

$ aws ec2 create-volume --availability-zone ap-northeast-1a --size 10 --volume-type gp2

표시되는 VolumeId를 기록해 둡니다.

MySQL Pod 작성

mysql.yaml

apiVersion: v1
kind: Pod
metadata:
  name: mysql
  labels:
    name: mysql
spec:
  containers:
    - resources:
        limits :
          cpu: 0.5
      image: mysql:5.6
      name: mysql
      env:
        - name: MYSQL_ROOT_PASSWORD
          value: yourpassword
      ports:
        - containerPort: 3306
          name: mysql
      volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
  volumes:
    - name: mysql-persistent-storage
      awsElasticBlockStore:
        volumeID: aws://ap-northeast-1a/｛上で作成した VolumeID｝
        fsType: ext4

포드 만들기

$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig create -f mysql.yaml
pod "mysql" created

$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig get pod
NAME        READY     STATUS    RESTARTS   AGE
mysql       1/1       Running   0          5m

MySQL Service 만들기

mysql-service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    name: mysql
  name: mysql
spec:
  ports:
    - port: 3306
  selector:
    name: mysql

서비스 만들기

$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig create -f mysql-service.yaml
service "mysql" created

$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig get svc
NAME         CLUSTER_IP   EXTERNAL_IP   PORT(S)    SELECTOR         AGE
kubernetes   10.3.0.1     <none>        443/TCP    <none>           1h
mysql        10.3.0.170   <none>        3306/TCP   name=mysql       5m

워드프레스 데이터용 EBS Volume 생성

$ aws ec2 create-volume --availability-zone ap-northeast-1a --size 10 --volume-type gp2

표시되는 VolumeId를 기록해 둡니다.

Wordpress Pod 만들기

wordpress.yaml

apiVersion: v1
kind: Pod
metadata:
  name: wordpress
  labels:
    name: wordpress
spec:
  containers:
    - image: wordpress
      name: wordpress
      env:
        - name: WORDPRESS_DB_PASSWORD
          value: yourpassword
      ports:
        - containerPort: 80
          name: wordpress
      volumeMounts:
        - name: wordpress-persistent-storage
          mountPath: /var/www/html
  volumes:
    - name: wordpress-persistent-storage
      awsElasticBlockStore:
        volumeID: aws://ap-northeast-1a/｛上で作成した VolumeID｝
        fsType: ext4

포드 만들기

$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig create -f wordpress.yaml
pod "mysql" wordpress

$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig get pod
NAME        READY     STATUS    RESTARTS   AGE
mysql       1/1       Running   0          8m
wordpress   1/1       Running   0          5m

Wordpress Service 만들기

wordpress-service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    name: wpfrontend
  name: wpfrontend
spec:
  ports:
    - port: 80
  selector:
    name: wordpress
  type: LoadBalancer

서비스 만들기

$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig create -f wordpress-service.yaml
service "mysql" created

$ kubectl --kubeconfig=clusters/kubernetes/kubeconfig get svc
NAME         CLUSTER_IP   EXTERNAL_IP   PORT(S)    SELECTOR         AGE
kubernetes   10.3.0.1     <none>        443/TCP    <none>           55m
mysql        10.3.0.170   <none>        3306/TCP   name=mysql       9m
wpfrontend   10.3.0.130                 80/TCP     name=wordpress   5m

Service를 작성할 때 type: LoadBalancer를 지정하면 자동으로 ELB가 작성되므로 확인

$ aws elb describe-load-balancers
{
    "LoadBalancerDescriptions": [
        {
            "Subnets": [
...snip...

표시된 DNSName에 브라우저로 액세스



Wordpress 설정 화면이 나타나면 확인

청소

kubernetes로 작성된 LoadBalancer(ELB), ELB용 SecurityGroup은 CloudFormation에서 관리되지 않으며 삭제되지 않으므로 수동으로 실행됩니다.

$ aws elb delete-load-balancer --load-balancer-name=｛LoadBalancerName｝

Volume도 마찬가지로 삭제

$ aws ec2 delete-volume --volume-id=｛VolumeID｝

destroy

$ ./kube-aws destroy
Destroyed cluster