kustomize: 비밀이 있는 경량 배포를 위한 Helm의 대안

It's recommended to structure a kustomize deployment into two folders base for the default deployment and overlays for the patching deployments.

The simplest deployment that infuses our secrets can look like the following:

base
├── deployment.yaml
├── kustomization.yaml
└── secrets.env

The deployment.yaml contains the kubernetes manifest that we'll use for the deployment.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  labels:
    app: my-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: ghcr.io/staubrein/myapp
        imagePullPolicy: Always
        env:
        - name: TOKEN
          valueFrom:
            secretKeyRef:
              name: my-app-secrets
              key: TOKEN

This manifest expects a secret my-app-secret with a key TOKEN and adds it to the container environment.

The content of this secret can be read from a casual .env file or in our case secrets.env .

TOKEN=XXXXXXXXXYXYYXYXYXYXYXYXYXYXXYXXXXXXXXYYY

To generate the secret from the file (and also system environment variables!) we first need to write the kustomize.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - deployment.yaml

secretGenerator:
- name: my-app-secrets
  env: secrets.env

This kustomize file takes the deployment.yaml as resource and combines it with generated secret from the secretGenerator .
It takes the values from the given secrets.env file into context and creates a secret with the given name my-app-secrets . It's important to import the file via the env key and not the file key, since file will turn the whole file into a secret while env makes all containing keys of the secrets.env accessible.

We can take a look at the finished manifests with kubectl kustomize base .

apiVersion: v1
data:
  TOKEN: |
    T888as8d8dja8sdjjajd8ahjjdknkasndanddnasd8a8sda8sd
kind: Secret
metadata:
  name: my-app-secrets-fmk84tgkh5
type: Opaque
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: my-app
  name: my-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - env:
        - name: TOKEN
          valueFrom:
            secretKeyRef:
              key: TOKEN
              name: my-app-secrets-fmk84tgkh5
        image: ghcr.io/staubrein/myapp
        imagePullPolicy: Always
        name: my-app

The secretGenerator of kustomize dynamically generated a secret from the secrets.env file with base64 encoded values and those are accessible within the container.

The best part on this solution, what makes it also very interesting for usage in pipelines or environments that don't use variables from files, is that it also works with system environment variables.

By not assigning a value to the key in a environment file, it will automatically try to read out the value for this key from the system environment. Let's demonstrate this in a simple example and add another key to the secrets.env .

TOKEN=XXXXXXXXXYXYYXYXYXYXYXYXYXYXXYXXXXXXXXYYY
FOO

It is important, that there is no equal sign after the key, otherwise the value will be considered empty and not overwritten with a system environment variable.
Let's fill this variable locally with export FOO=bar and generate the secret with kustomize by running kubectl kustomize base .

apiVersion: v1
data:
  FOO: YmFy
  TOKEN: WFhYWFhYWFhYWVhZWVhZWFlYWVhZWFlYWVhZWFhZWFhYWFhYWFhZWVk=
kind: Secret
metadata:
  name: dailyottr-secrets-h7b26g7c74
  namespace: my-app
type: Opaque

As seen in the output above, it set the variable FOO with the value we set in the system environment variable :)

The deployment can be triggered via kubectl kustomize base | kubectl apply -f - .

To fully show the potential of kustomize, especially the functionality of merging and overwriting different environments, I'll write another article soon that patches the base deployment.