Docker, Kubernetes, and Co.

Prepare hosts

DNS Setup

root@master-node:~# cat /etc/hosts localhost master-node slave-node1 slave-node2 slave-node3

# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

Add kernel modules

root@master-node:~# cat /etc/modules
# /etc/modules: kernel modules to load at boot time.

Install packages for VirtualBox additions

apt-get install linux-headers-$(uname -r)

apt-get install build-essential

Install VirtualBox additions after that.

Install NFS packages

Server (master-node)

apt install nfs-kernel-server

Worker nodes

apt install nfs-common

Configure NFS shares

Create volumes/directories to be shared

root@master-node:~# mkdir -p /var/nfs/vol
root@master-node:~# mkdir -p /var/nfs/vol1
root@master-node:~# mkdir -p /var/nfs/vol2
root@master-node:~# mkdir -p /var/nfs/vol3

Configure shares

root@master-node:~# cat /etc/exports

Configure service

root@master-node:~# systemctl is-enabled nfs-server
root@master-node:~# systemctl restart nfs-server

Verify access from the server and from the worker nodes

root@master-node:~# showmount --exports
Export list for
root@slave-node1:~# showmount --exports
Export list for

Start installing Kubernetes cluster



To install MetalLB, apply the manifest:

kubectl apply -f
kubectl apply -f

Then add the configmap

root@master-node:~# cat k0s/manifests/layer2-config.yaml
apiVersion: v1
kind: ConfigMap
namespace: metallb-system
name: config
config: |
- name: my-ip-space
protocol: layer2

Check the status

root@master-node:~# kubectl get svc -o wide
kubernetes ClusterIP <none> 443/TCP 42h <none>
my-nginx LoadBalancer 80:31001/TCP 165m app=nginx
nginx-deployment NodePort <none> 80:30598/TCP 25h app=nginx

Load balanced nginx service can be now accessed from the host network, like this:

root@master-node:~# curl
<!DOCTYPE html>
<title>Welcome to nginx!</title>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href=""></a>.<br/>
Commercial support is available at
<a href=""></a>.</p>

<p><em>Thank you for using nginx.</em></p>

NFS Provisioner Operator

Install Helm

Kubernetes NFS Subdir External Provisioner

root@master-node:~# helm repo add nfs-subdir-external-provisioner
"nfs-subdir-external-provisioner" has been added to your repositories

root@master-node:~# helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner --set nfs.server= --set nfs.path=/var/nfs/vol1
NAME: nfs-subdir-external-provisioner
LAST DEPLOYED: Thu Apr 28 16:17:19 2022
NAMESPACE: default
STATUS: deployed

Configure POD to use the external NFS Provisioner

Vagualy based on

Crucial element is in the PV Claim definition:

root@master-node:~/k0s/manifests# cat pv-claim.yaml
apiVersion: v1
kind: PersistentVolumeClaim
  name: task-pv-claim
  storageClassName: nfs-client
    - ReadWriteOnce
      storage: 1Gi

Note storageClassName attribute nfs-client, it is the same as the storageClass definition we created in the previous step.

Install PostgreSQL cluster

PostgreSQL Operator

Start with the instruction from up to the point where you are ready to deploy the first PostgreSQL cluster via the GUI. Then use GUI to create a skeleton of the deployment YAML.

Create cluster

Start with the GUI:

More instances will create more standby pods and probably are not needed. Also add user(s) and database(s):

But don’t hit Create Cluster button, use Copy instead and put it into a YAML file:

root@master-node:~/k0s/manifests# vi pgtest4.yaml
kind: "postgresql"
apiVersion: ""

  name: "acid-pgtest4"
  namespace: "default"
    team: acid

  teamId: "acid"
    version: "14"
  numberOfInstances: 2
  enableMasterLoadBalancer: true
  enableConnectionPooler: false
    size: "20Gi"
    storageClass: nfs-client
    camunda: []
    camunda: camunda
    # IP ranges to access your cluster go here

      cpu: 100m
      memory: 100Mi
      cpu: 500m
      memory: 500Mi

Two critical attributes cannot be added via GUI. storageClass: nfs-clinet will make sure that the PV Claims created request volumes from the NFS Provisioner and allowedSourceRanges: will allow connection to the database cluster from any IP. If not specified, it will allow connections only from the localhost. This can be refined later.

Apply the created YAML file:

root@master-node:~/k0s/manifests# kubectl apply -f pgtest4.yaml created

And after a while observe your new PostgreSQL cluster be created:

root@master-node:~/k0s/manifests# kubectl get svc -w
NAME                   TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)          AGE
acid-pgtest2           LoadBalancer   5432:32015/TCP   16h
acid-pgtest2-config    ClusterIP      None             <none>          <none>           16h
acid-pgtest2-pooler    ClusterIP    <none>          5432/TCP         16h
acid-pgtest2-repl      ClusterIP     <none>          5432/TCP         16h
acid-pgtest3           LoadBalancer   5432:32518/TCP   13h
acid-pgtest3-config    ClusterIP      None             <none>          <none>           13h
acid-pgtest3-repl      ClusterIP    <none>          5432/TCP         13h
acid-pgtest4           LoadBalancer   5432:32511/TCP   2m6s
acid-pgtest4-config    ClusterIP      None             <none>          <none>           2m
acid-pgtest4-repl      ClusterIP     <none>          5432/TCP         2m6s
kubernetes             ClusterIP        <none>          443/TCP          2d17h
my-nginx               LoadBalancer   80:31001/TCP     25h
nginx-deployment       NodePort    <none>          80:30598/TCP     2d
postgres-operator      ClusterIP    <none>          8080/TCP         17h
postgres-operator-ui   ClusterIP   <none>          80/TCP           17h

Our new Service is now up and available on the IP Let’s check it.
We will need a password for our user camunda. We can do the following:

root@master-node:~/k0s/manifests# export PGPASSWORD=$(kubectl get secret -o 'jsonpath={.data.password}' | base64 -d)               
root@master-node:~/k0s/manifests# echo $PGPASSWORD

In the first line above camunda is the user name and acid-pgtest4 is the name of the service. With the above we set an environment variable, which will be used in psql command:

root@master-node:~/k0s/manifests# psql -U camunda -h
psql (13.5 (Debian 13.5-0+deb11u1), server 14.2 (Ubuntu 14.2-1.pgdg18.04+1))
WARNING: psql major version 13, server major version 14.
         Some psql features might not work.
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.

camunda=> \l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
 camunda   | camunda  | UTF8     | en_US.utf-8 | en_US.utf-8 |
 postgres  | postgres | UTF8     | en_US.utf-8 | en_US.utf-8 |
 template0 | postgres | UTF8     | en_US.utf-8 | en_US.utf-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.utf-8 | en_US.utf-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
(4 rows)


The database can be now accessed from the host machine as well:

Camunda Engine

Install via Helm chart

Starting point:

Secrete creation differs a bit, different user and also the password, which we already setup for PostgreSQL access in the previous part.

root@master-node:~# kubectl create secret generic                   \
    camunda-bpm-platform-db-credentials             \
    --from-literal=DB_USERNAME=camunda \
secret/camunda-bpm-platform-db-credentials created

Number of instances can be adjusted in the values.yaml. The file will look like this:

root@master-node:~# cat k0s/manifests/values.yaml
# Default values for camunda-bpm-platform.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

  debug: false
  replicaCount: 3
  nameOverride: ""
  fullnameOverride: ""

  repository: camunda/camunda-bpm-platform
  tag: latest
  pullPolicy: IfNotPresent
  pullSecrets: []
  command: []
  args: []

# By default H2 database is used, which is handy for demos and tests,
# however, H2 is not supported in a clustered scenario.
# So for real-world workloads, an external database like PostgreSQL should be used.
  driver: org.postgresql.Driver
  url: jdbc:postgresql://acid-pgtest4:5432/camunda
  credentialsSecretName: camunda-bpm-platform-db-credentials

  type: LoadBalancer
  port: 8080
  portName: http

  enabled: false
    type: ClusterIP
    port: 9404
    portName: metrics
    annotations: "true" "/" "9404"


Note that the service type has to be set to LoadBalancer if we deploy more than one instance. This has one caveat though. With the current Helm chart, it is not possible to specify the sessionAffinity attribute. If we don’t do that, the web interface will behave somewhat weirdly. For example, you won’t be able to disable the Telemetry consent pop-up or won’t be able to log in at all. Obviously, the Helm chart has to be adjusted, but the short-term solution would be to patch the service configuration post-install. In my case, I was able to accomplish this directly from Lens.

Here is an excerpt from its manifest after the patching:

Confirm we can access the engine

To be continued…

What we have achieved so far. We have installed a bare metal Kubernetes cluster with k0s, we have a working PostgreSQL cluster and a working Camunda 7.17 installation.
Let’s fill it with life and continue to the next article Docker, Kubernetes, and Co. – part II

You may also want to take a look at one of the previously published articles as we will be doing something similar: Camunda on Oracle Cloud – will it work?



Leave a Reply