Configuration

Specify environment variables for the container

The configuration of the application in a container is often communicated through environment variables (abbreviated env).

Kubernetes provides us with a way to pass a list of envs inside each Pod container. For example, we have an image 080196/hello-env with the following code:

const http = require("http");

const server = http.createServer((req, res) => {
  console.log("Hello env\n")
});

server.listen(process.env.PORT, () => {
  console.log("Server listen on port ", process.env.PORT)
})

In this image we have the PORT configuration of the application that will be passed inside the container through env PORT. Create a file named pod-hello-env.yaml with the following configuration:

apiVersion: v1
kind: Pod
metadata:
  name: hello-env
spec:
  containers:
    - image: 080196/hello-env
      name: hello-env
      ports:
        - containerPort: 3000
      env: # pass env to container
        - name: PORT # env name
          value: "3000" # env value

In this config file, we have specified an attribute named env, this is a Pod attribute that helps us pass env inside the container, with the name of env being PORT and its value will be 3000, note that the value env value is always a string , if you leave it as a number it will give an error. Now we create the Pod and check the log.

$ kubectl apply -f pod-hello-env.yaml
pod/hello-env created
$ kubectl logs hello-env
Server listen on port 3000

Here, our application runs correctly according to the PORT we passed into the container through env. You can check more closely by accessing the container and printing out its env as follows:

$ kubectl exec -it hello-env -- sh
/app # env | grep PORT
KUBERNETES_PORT=tcp://10.96.0.1:443
KUBERNETES_SERVICE_PORT=443
PORT=3000 // This is our env
KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443
KUBERNETES_SERVICE_PORT_HTTPS=443
/app # exit

Note that this env list will not be able to update inside the container when that container is already running. To update, we must delete the Pod to get it running again.

Using env to pass configuration inside the container is very easy, but as our application gets larger and needs more and more env, we will realize that specifying env directly in the Pod like this has many limitations. Firstly, our config file will be extremely long, secondly, if there are some envs we will use over and over again, we will copy and paste them in many places. For example:

apiVersion: v1
kind: Pod
metadata:
  name: hello-env
spec:
  containers:
    - image: 080196/hello-env
      name: hello-env
      ports:
        - containerPort: 3000
      env:
        - name: PORT
          value: "3000"
        - name: POSTGRES_DB
          value: postgres
        - name: POSTGRES_USER
          value: postgres
        - name: POSTGRES_PASSWORD
          value: postgres

---
apiVersion: v1
kind: Pod
metadata:
  name: postgres
spec:
  containers:
    - image: postgres
      name: postgres
      ports:
        - containerPort: 5432
      env:
        - name: POSTGRES_DB
          value: postgres
        - name: POSTGRES_USER
          value: postgres
        - name: POSTGRES_PASSWORD
          value: postgres

In this config file we have 2 Pods, one Pod runs the application and one Pod runs the database. We will see that the configuration related to database information will be repeated twice. When our application grows, it will need a lot more envs. At this point, we just want to declare the general configuration in one place, and use that configuration to reuse many times. Then kubernetes provides us with a resource to do that, ConfigMap.

Note that information about the user and password of the database is sensitive data. Usually with sensitive information we will not use ConfigMap but will use Secret (will be discussed below), here for example. so we will ConfigMap.

ConfigMap

This is a resource that helps us separate configuration. The value will be defined as key/value pairs in the data attribute, as follows:

data:
    <key-1>: <value-1>
    <key-2>: <value-2>

And this value will be passed inside the container as an env. And because ConfigMap is an individual resource, we can reuse it for many different containers. Using ConfigMap is a way to avoid having to write the env configuration inside the Pod container config.

Create a ConfigMap

Now we will create a ConfigMap. Create a cm-db.yaml file with the following configuration:

apiVersion: v1
kind: ConfigMap
metadata:
  name: postgres-config
data:
  DB: postgres
  USER: postgres
  PASSWORD: postgres

Here we will create a ConfigMap named postgres-config with 3 keys: DB, USER, PASSWORD.

$ kubectl apply -f cm-db.yaml
configmap/postgres-config created

Or we can also create ConfigMap through the cli command without writing a config file, as follows:

$ kubectl create cm postgres-config --from-literal=DB=postgres --from-literal=USER=postgres --from-literal=PASSWORD=postgres
configmap/postgres-config created

Pass ConfigMap inside the container. We will reuse the example above, create a file named pod-with-cm.yaml using image 080196/hello-cm , the image code is as follows:

const http = require("http");
const { Client } = require("pg");

const { DB_HOST, POSTGRES_USER, POSTGRES_DB, POSTGRES_PASSWORD } = process.env;

const client = new Client({
  host: DB_HOST,
  user: POSTGRES_USER,
  database: POSTGRES_DB,
  password: POSTGRES_PASSWORD,
  port: 5432,
});
client.connect().then(() => {
  console.log("connect db successfully");
})

const server = http.createServer((req, res) => {
  console.log("Hello env\n")
});

server.listen(process.env.PORT, () => {
  console.log("Server listen on port ", process.env.PORT)
})

Config of pod-with-cm.yaml file:

apiVersion: v1
kind: Pod
metadata:
  name: hello-cm
  labels:
    app: application
spec:
  containers:
    - image: 080196/hello-cm
      name: hello-cm
      ports:
        - containerPort: 3000
      envFrom: # using envFrom instead of env
        - configMapRef: # referencing the ConfigMap
            name: postgres-config # name of the ConfigMap
          prefix: POSTGRES_ # All environment variables will be prefixed with POSTGRES_
      env:
        - name: PORT
          value: "3000"
        - name: DB_HOST
          value: postgres

---
apiVersion: v1
kind: Pod
metadata:
  name: postgres
  labels:
    app: db
spec:
  containers:
    - image: postgres
      name: postgres
      ports:
        - containerPort: 5432
      envFrom:
        - configMapRef:
            name: postgres-config
          prefix: POSTGRES_

---
apiVersion: v1
kind: Service
metadata:
  name: postgres
  labels:
    app: db
spec:
  selector:
    app: db
  ports:
    - port: 5432
      targetPort: 5432

Here we will use the envFrom attribute instead of env to specify the use of ConfigMap, the prefix field will be concatenated in front of all our envs with the value POSTGRES_. As you can see, instead of having to rewrite env in both Pods, we just need to create a configmap and use it many times. Let's create and test:

$ kubectl apply -f pod-with-cm.yaml -l app=db
pod/postgres created
service/postgres created

Here we check that the Pod db is running, then let's create a hello-cm pod and test it:

$ kubectl get pod
pod/hello-cm created
$ kubectl apply -f pod-with-cm.yaml -l app=application
pod/hello-cm created
$ kubectl logs hello-cm
Server listen on port  3000
connect db successfully

If we log in and print out the line connect db successfully , then we have successfully passed the configuration to the container through ConfigMap. At this point, we know how to pass configuration through env to the container using ConfigMap. But what if our application's configuration is a config file? We can also use ConfigMap.

Use ConfigMap to transfer configuration files into the container through volume config

With ConfigMap we can use a key whose value is the content of an entire config file, as follows:

data:
    <key>: |
        line
        line

For example, we will create a config file for nginx that compresses its responses before returning it to the client, creating a file called nginx-config-cm.yaml with the following config:

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-config
data:
  my-nginx-config.conf: |
    server {
      listen 80;
      server_name www.kubia-example.com;

      gzip on;
      gzip_types text/plain application/xml;

      location / {
        root /usr/share/nginx/html;
        index index.html index.htm;
      }
    }

Here we create a ConfigMap with the name nginx-config, with the key my-nginx-config.conf which will be the name of the file when we pass it into the container. Create a file named pod-nginx.yaml with the following configuration:

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
    - image: nginx:alpine
      name: web-server
      volumeMounts:
        - mountPath: /etc/nginx/conf.d # mount content of the configmap to container
          name: config
          readOnly: true
  volumes:
    - name: config # volume use configmap as content
      configMap:
        name: nginx-config # configmap name

Here, our ConfigMap will be used as a volume, with content being the content in the ConfigMap. At this time, our volume will contain a file named my-nginx-config.conf, then this file will be mounted to the web-server container in the /etc/nginx/conf.d folder, we will have /etc/nginx/ conf.d/my-nginx-config.conf

Let's create a Pod and check if our config is correct:

$ kubectl apply -f nginx-config-cm.yaml
configmap/nginx-config created
$ kubectl apply -f pod-nginx.yaml
pod/nginx created
$ kubectl port-forward nginx 8080:80
Forwarding from 127.0.0.1:8080 -> 80
Forwarding from [::1]:8080 -> 80

Open another terminal:

$ curl -H "Accept-Encoding: gzip" -I localhost:8080
HTTP/1.1 200 OK
Server: nginx/1.21.3
Date: Fri, 24 Sep 2021 09:58:47 GMT
Content-Type: text/html
Last-Modified: Tue, 07 Sep 2021 15:50:58 GMT
Connection: keep-alive
ETag: W/"61378a62-267"
Content-Encoding: gzip // response is compressed

We can check more closely by accessing the Pod:

$ kubectl exec -it nginx -- sh
/ # cd /etc/nginx/conf.d/
/etc/nginx/conf.d # ls
my-nginx-config.conf

Here it is, this is the file that we mounted into the container via ConfigMap. Let's print its content and see:

/etc/nginx/conf.d # cat my-nginx-config.conf 
server {
  listen 80;
  server_name www.kubia-example.com;

  gzip on;
  gzip_types text/plain application/xml;

  location / {
    root /usr/share/nginx/html;
    index index.html index.htm;
  }
}
/etc/nginx/conf.d # exit

This is the value of the my-nginx-config.conf key that we defined in ConfigMap. We can also change the name of the file by writing config in the volume as follows:

...
  volumes:
    - name: config
      configMap:
        name: nginx-config
        items:
          - key: my-nginx-config.conf
            path: gzip.conf # change name of file from my-nginx-config.conf to gzip.conf

Here we rename the file from my-nginx-config.conf to gzip.conf using the path attribute. We can also just mount the content of the file:

...
  containers:
    - image: nginx:alpine
      name: web-server
      volumeMounts:
        - name: config
          mountPath: /etc/nginx/conf.d/gzip.conf # mount content of my-nginx-config.conf file to gzip.conf file
          subPath: my-nginx-config.conf # Instead of mounting the whole volume, you’re only mounting the my-nginx-config.conf
          readOnly: true
...

Here, we copy the content from the file my-nginx-config.conf to gzip.conf. If you want to specify file permissions, use the defaultMode property.

...
  volumes:
    - name: config # volume use configmap as content
      configMap:
        name: nginx-config # configmap name
        defaultMode: "0600"

When we use ConfigMap in the form of volume config. If we change the value of ConfigMap, it will automatically referencing and updating inside the Pod's volume, and we need the application to detect changes in our config file to reload the new config and re-run. If we use ConfigMap in env form, it will not update automatically, we need to re-create the Pod.

We use ConfigMap when we want to use config in many different Pods and the data is not sensitive data. For sensitive data, kubernetes provides us with another resource called Secret .

Secret

Secret is similar to ConfigMap, data is stored as key/value pairs, the way we use Secret is similar to ConfigMap. Secret is different from ConfigMap in that it is used to contain sensitive data. For ConfigMap, developers who can access our kubernetes cluster can read it, but for Secret, this data is not readable by everyone, right? Only with permissions granted by the administrator can we read.

In addition, kubernetes will increase security a bit by only sending secrets to worker nodes that have a Pod that needs to use it, and the Secret data will be stored in memory and never in physical storage, this will help when workers If the node dies, the Secret in memory will be deleted. And at the master node, the Secret will be saved under etcd in encrypted form.

Create a Secret

To create a Secret, we should use CLI rather than creating a config file. Back to the DB example above, instead of using ConfigMap, we will use Secret.

$ kubectl create secret generic postgres-config --from-literal=DB=postgres --from-literal=USER=postgres --from-literal=PASSWORD=postgres
secret/postgres-config created

When we look at the secret information, we will see that it is saved in base64 encoded format.

$ kubectl get secret postgres-config -o yaml
apiVersion: v1
data: // base64 encode.
  DB: cG9zdGdyZXM=
  PASSWORD: cG9zdGdyZXM=
  USER: cG9zdGdyZXM=
kind: Secret
metadata:
  creationTimestamp: "2021-09-24T10:40:26Z"
  name: postgres-config
  namespace: default
  resourceVersion: "1027803"
  uid: 9c7ed846-2ca4-4c58-946a-26cdccce2b1f
type: Opaque

Use Secret in Pod.

apiVersion: v1
kind: Pod
metadata:
  name: hello-cm
  labels:
    app: application
spec:
  containers:
    - image: 080196/hello-cm
      name: hello-cm
      ports:
        - containerPort: 3000
      envFrom: # using envFrom instead of env
        - secretRef: # use secretRef instead of env configMapRef
            name: postgres-config # name of the Secret
          prefix: POSTGRES_
      env:
        - name: PORT
          value: "3000"
        - name: DB_HOST
          value: postgres

---
apiVersion: v1
kind: Pod
metadata:
  name: postgres
  labels:
    app: db
spec:
  containers:
    - image: postgres
      name: postgres
      ports:
        - containerPort: 5432
      envFrom:
        - secretRef:
            name: postgres-config
          prefix: POSTGRES_

---
apiVersion: v1
kind: Service
metadata:
  name: postgres
  labels:
    app: db
spec:
  selector:
    app: db
  ports:
    - port: 5432
      targetPort: 5432

Here we will use the secretRef attribute instead of configMapRef. When we pass the Secret to the Pod it will be decoded from base64 form back to the same.

Remember:

• Use ConfigMap for non-sensitive, plain configuration information.

• Use Secret for sensitive information, use CLI, not config file.