Fixes: #236 - Running Zammad with replicas > 1 (#243)

- Splits up the chart from one StatefulSet into 4 Deployments and one Job. - The nginx and railsserver Deployments are freely scalable, the scheduler and websocket must remain at replicas: 1 - The Job will be re-created on any chart update (via uuid in the name) and run the migrations. Deployments will fail until migrations are executed. This greatly reduces start-up and update downtime. - Refactor some code into helper templates to reduce redundancy / improve maintainability. - Storage requirements changed. Please read the updating instructions carefully before updating. Special thanks to @klml and @monotek for guidance and feedback.
zammad · Apr 26, 2024 · 0877f7d · 0877f7d
1 parent fa6777b
commit 0877f7d
Show file tree

Hide file tree

Showing 19 changed files with 574 additions and 557 deletions.
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
@@ -124,8 +124,8 @@ jobs:
       - name: Create Namespace 'zammad'
         run: kubectl create namespace zammad
 
-      - name: Install secrets
-        run: kubectl create --namespace zammad --filename zammad/ci/full-secrets.yaml
+      - name: Install additional objects for 'full' test scenario
+        run: kubectl create --namespace zammad --filename zammad/ci/full-objects.yaml
 
       - name: Run chart-testing (install)
         run: ct install --config .github/ct.yaml --helm-extra-args '--timeout 900s'
diff --git a/zammad/Chart.yaml b/zammad/Chart.yaml
@@ -1,6 +1,6 @@
 apiVersion: v2
 name: zammad
-version: 11.0.0
+version: 12.0.0
 appVersion: 6.3.0
 description: Zammad is a web based open source helpdesk/customer support system with many features to manage customer communication via several channels like telephone, facebook, twitter, chat and e-mails.
 home: https://zammad.org

diff --git a/zammad/README.md b/zammad/README.md
@@ -31,6 +31,15 @@ helm repo add zammad https://zammad.github.io/zammad-helm
 helm upgrade --install zammad zammad/zammad
 ```
 
+Once the Zammad pod is ready, it can be accessed using the ingress or port forwarding.
+To use port forwarding:
+
+```console
+kubectl port-forward service/zammad-nginx 8080
+```
+
+Now you can open <http://localhost:8080> in your browser.
+
 ## Uninstalling the Chart
 
 To remove the chart again use the following:
@@ -58,15 +67,8 @@ Only if you have a large volume of tickets and attachments, you may need to stor
 
 We recommend the `S3` storage provider using the optional `minio` subchart in this case.
 
-You can also use `File` storage. In this case, you should check:
-
-- If you already use an `externalVolumeClaim` with `ReadWriteMany` access, you can keep using that.
-- If you already use an `externalVolumeClaim` with another access mode, we recommend migrating to S3 storage (see below).
-- If you used the default `PVC` of the Zammad `StatefulSet`, we also recommend migrating to S3 storage (see below).
-
-Background information: a future version of Zammad will increase the scalability by splitting up the current `Statefulset`
-into several `Deployment`s which can be scaled. This means the `PVC` of the current `StatefulSet` will then not be usable any more,
-and any volumes will have to support `ReadWriteMany` access.
+You can also use `File` storage. In this case, you need to provide an existing `PVC` via `zammadConfig.storageVolume`.
+Note that this `PVC` must provide `ReadWriteMany` access to work properly for the different Deployments which may be on different nodes.
 
 #### How to migrate from `File` to `S3` storage
 
@@ -102,11 +104,11 @@ zammadConfig:
     zammad:
       securityContext:
         runAsUser: null
-      customInit: |
-        # use an openshift uid owned /tmp for attachments upload
-        mkdir -pv /opt/zammad/var/tmp && chmod -v +t /opt/zammad/var/tmp
-  railsserver:
-    tmpdir: "/opt/zammad/var/tmp"
+  volumePermissions:
+    enabled: false
+  tmpDirVolume:
+    emptyDir:
+      medium: memory
 
 elasticsearch:
   sysctlImage:
@@ -142,18 +144,26 @@ redis:
       enabled: false
 ```
 
-## Using Zammad
+## Upgrading
 
-Once the Zammad pod is ready, it can be accessed using the ingress or port forwarding.
-To use port forwarding:
+### From Chart Version 11.x to 12.0.0
 
-```console
-kubectl port-forward service/zammad 8080
-```
+#### The Previous `StatefulSet` Was Split up into `Deployments`
 
-Now you can open <http://localhost:8080> in your browser.
+- `replicas` can be set independently now for `zammad-nginx` and `zammad-railsserver`, allowing free scaling and HA setup for these.
+  - For `zammad-scheduler` and `zammad-websocket`, `replicas` is fixed to `1` as they may only run once in the cluster.
+- The `initContainers` moved to a new `zammad-init` `Job` now which will be run on every `helm upgrade`. This reduces startup time greatly.
+- The nginx `Service` was renamed from `zammad` to `zammad-nginx`.
+- The previous `Values.sidecars` setting does not exist any more. Instead, you need to specify sidecars now on a per deployment basis, e.g. `Values.zammadConfig.scheduler.sidecars`.
 
-## Upgrading
+#### Storage Requirements Changed
+
+- If you use the default `DB` or the new `S3` storage backend for file storage, you don't need to do anything.
+- If you use the `File` storage backend instead, Zammad now requires a `ReadWriteMany` volume for `storage/` that is shared in the cluster.
+  - If you already had one via `persistence.existingClaim` before, you need to ensure it has `ReadWriteMany` access to be mountable across nodes and provide it via `zammadConfig.storageVolume.existingClaim`.
+  - If you used the default `PersistentVolumeClaim` of the `StatefulSet`, you need to take manual action:
+    - You can either migrate to `S3` storage **before upgrading** to the new major version as described above in [Configuration](#how-to-migrate-from-file-to-s3-storage).
+    - Or you can provide a `zammadConfig.storageVolume.existingClaim` with `ReadWriteMany` permission and migrate your existing data to it from the old `StatefulSet`.
 
 ### From Chart Version 10.x to 11.0.0
 

diff --git a/zammad/ci/full-secrets.yaml → zammad/ci/full-objects.yaml b/zammad/ci/full-secrets.yaml → zammad/ci/full-objects.yaml
@@ -37,4 +37,15 @@ data:
 kind: Secret
 metadata:
   name: autowizard
-type: Opaque
+type: Opaque
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: storage-volume-claim
+spec:
+  accessModes:
+    - ReadWriteOnce   # Testing env does not provide ReadWrite Many, but for CI this is enough.
+  resources:
+    requests:
+      storage: 32Mi
diff --git a/zammad/ci/full-values.yaml b/zammad/ci/full-values.yaml
@@ -33,5 +33,8 @@ redis:
     existingSecretPasswordKey: redis-password
 
 zammadConfig:
+  storageVolume:
+    enabled: true
+    existingClaim: 'storage-volume-claim'
   minio:
     enabled: true
diff --git a/zammad/templates/_helpers.tpl b/zammad/templates/_helpers.tpl
@@ -135,3 +135,169 @@ S3 access URL
 {{- end -}}
 {{- end -}}
 {{- end -}}
+
+{{/*
+environment variables for the Zammad Rails stack
+*/}}
+{{- define "zammad.env" -}}
+{{- if or .Values.zammadConfig.redis.pass .Values.secrets.redis.useExisting -}}
+- name: REDIS_PASSWORD
+  valueFrom:
+    secretKeyRef:
+      name: {{ template "zammad.redisSecretName" . }}
+      key: {{ .Values.secrets.redis.secretKey }}
+{{- end }}
+- name: MEMCACHE_SERVERS
+  value: "{{ if .Values.zammadConfig.memcached.enabled }}{{ .Release.Name }}-memcached{{ else }}{{ .Values.zammadConfig.memcached.host }}{{ end }}:{{ .Values.zammadConfig.memcached.port }}"
+- name: RAILS_TRUSTED_PROXIES
+  value: "{{ .Values.zammadConfig.railsserver.trustedProxies }}"
+- name: REDIS_URL
+  value: "redis://:$(REDIS_PASSWORD)@{{ if .Values.zammadConfig.redis.enabled }}{{ .Release.Name }}-redis-master{{ else }}{{ .Values.zammadConfig.redis.host }}{{ end }}:{{ .Values.zammadConfig.redis.port }}"
+- name: POSTGRESQL_PASS
+  valueFrom:
+    secretKeyRef:
+      name: {{ template "zammad.postgresqlSecretName" . }}
+      key: {{ .Values.secrets.postgresql.secretKey }}
+- name: DATABASE_URL
+  value: "postgres://{{ .Values.zammadConfig.postgresql.user }}:$(POSTGRESQL_PASS)@{{ if .Values.zammadConfig.postgresql.enabled }}{{ .Release.Name }}-postgresql{{ else }}{{ .Values.zammadConfig.postgresql.host }}{{ end }}:{{ .Values.zammadConfig.postgresql.port }}/{{ .Values.zammadConfig.postgresql.db }}?{{ .Values.zammadConfig.postgresql.options }}"
+{{ include "zammad.env.S3_URL" . }}
+- name: TMP # All zammad containers need the possibility to create temporary files, e.g. for file uploads or image resizing.
+  value: {{ .Values.zammadConfig.railsserver.tmpdir }}
+{{- with .Values.extraEnv }}
+{{ toYaml . }}
+{{- end }}
+{{- if .Values.autoWizard.enabled }}
+- name: AUTOWIZARD_RELATIVE_PATH
+  value: tmp/auto_wizard/auto_wizard.json
+{{- end }}
+{{- end -}}
+
+{{/*
+environment variable to let Rails fail during startup if migrations are pending
+*/}}
+{{- define "zammad.env.failOnPendingMigrations" -}}
+# Let containers fail if migrations are pending.
+- name: RAILS_CHECK_PENDING_MIGRATIONS
+  value: 'true'
+{{- end -}}
+
+{{/*
+volume mounts for the Zammad Rails stack
+*/}}
+{{- define "zammad.volumeMounts" -}}
+- name: {{ template "zammad.fullname" . }}-tmp
+  mountPath: /tmp
+- name: {{ template "zammad.fullname" . }}-tmp
+  mountPath: /opt/zammad/tmp
+{{- if .Values.zammadConfig.storageVolume.enabled }}
+- name: {{ template "zammad.fullname" . }}-storage
+  mountPath: /opt/zammad/storage
+{{- end -}}
+{{- if .Values.autoWizard.enabled }}
+- name: autowizard
+  mountPath: "/opt/zammad/tmp/auto_wizard"
+{{- end }}
+{{- end -}}
+
+{{/*
+volumes for the Zammad Rails stack
+*/}}
+{{- define "zammad.volumes" -}}
+- name: {{ include "zammad.fullname" . }}-tmp
+  {{- toYaml .Values.zammadConfig.tmpDirVolume | nindent 2 }}
+{{- if .Values.zammadConfig.storageVolume.enabled }}
+{{- if .Values.zammadConfig.storageVolume.existingClaim }}
+- name: {{ template "zammad.fullname" . }}-storage
+  persistentVolumeClaim:
+    claimName: {{ .Values.zammadConfig.storageVolume.existingClaim | default (include "zammad.fullname" .) }}
+{{- else }}
+  {{ fail "Please provide an existing PersistentVolumeClaim with ReadWriteMany access if you enable .Values.zammadConfig.storageVolume." }}
+{{- end -}}
+{{- end -}}
+{{- if .Values.autoWizard.enabled }}
+- name: autowizard
+  secret:
+    secretName: {{ template "zammad.autowizardSecretName" . }}
+    items:
+    - key: {{ .Values.secrets.autowizard.secretKey }}
+      path: auto_wizard.json
+{{- end }}
+{{- end -}}
+
+{{/*
+shared configuration for Zammad Pods
+*/}}
+{{- define "zammad.podSpec" -}}
+{{- with .Values.image.imagePullSecrets }}
+imagePullSecrets:
+  {{- toYaml . | nindent 2 }}
+{{- end }}
+{{- if .Values.serviceAccount.create }}
+serviceAccountName: {{ include "zammad.serviceAccountName" . }}
+{{- end }}
+{{- with .Values.nodeSelector }}
+nodeSelector:
+  {{- toYaml . | nindent 2 }}
+{{- end }}
+{{- with .Values.affinity }}
+affinity:
+  {{- toYaml . | nindent 2 }}
+{{- end }}
+{{- with .Values.tolerations }}
+tolerations:
+  {{- toYaml . | nindent 2 }}
+{{- end }}
+{{- with .Values.securityContext }}
+securityContext:
+  {{- toYaml . | nindent 2 }}
+{{- end }}
+{{- end -}}
+
+{{/*
+shared configuration for Zammad Deployment Pods
+*/}}
+{{- define "zammad.podSpec.deployment" -}}
+{{ include "zammad.podSpec" . }}
+{{- if .Values.zammadConfig.initContainers.volumePermissions.enabled }}
+initContainers:
+  - name: zammad-volume-permissions
+    image: "{{ .Values.zammadConfig.initContainers.volumePermissions.image.repository }}:{{ .Values.zammadConfig.initContainers.volumePermissions.image.tag }}"
+    imagePullPolicy: {{ .Values.zammadConfig.initContainers.volumePermissions.image.pullPolicy }}
+    command:
+      {{- .Values.zammadConfig.initContainers.volumePermissions.command | toYaml | nindent 6 }}
+    {{- with .Values.zammadConfig.initContainers.volumePermissions.resources }}
+    resources:
+      {{- toYaml . | nindent 6 }}
+    {{- end }}
+    {{- with .Values.zammadConfig.initContainers.volumePermissions.securityContext }}
+    securityContext:
+      {{- toYaml . | nindent 6 }}
+    {{- end }}
+    volumeMounts:
+      {{- include "zammad.volumeMounts" . | nindent 6 }}
+{{- end }}
+{{- end -}}
+
+{{/*
+shared configuration for Zammad containers
+*/}}
+{{- define "zammad.containerSpec" -}}
+image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
+imagePullPolicy: {{ .Values.image.pullPolicy }}
+{{- with .containerConfig.livenessProbe }}
+livenessProbe:
+  {{- toYaml . | nindent 2 }}
+{{- end }}
+{{- with .containerConfig.readinessProbe }}
+readinessProbe:
+  {{- toYaml . | nindent 2 }}
+{{- end }}
+{{- with .containerConfig.resources }}
+resources:
+  {{- toYaml . | nindent 2 }}
+{{- end }}
+{{- with .containerConfig.securityContext }}
+securityContext:
+  {{- toYaml . | nindent 2 }}
+{{- end }}
+{{- end -}}
diff --git a/zammad/templates/configmap-nginx.yaml b/zammad/templates/configmap-nginx.yaml
@@ -13,11 +13,11 @@ data:
     server_tokens off;
 
     upstream zammad-railsserver {
-        server localhost:3000;
+        server {{ template "zammad.fullname" . }}-railsserver:3000;
     }
 
     upstream zammad-websocket {
-        server localhost:6042;
+        server {{ template "zammad.fullname" . }}-websocket:6042;
     }
 
     server {
@@ -106,9 +106,9 @@ data:
     }
   nginx.conf: |-
     worker_processes auto;
-    
+
     pid /tmp/nginx.pid;
-    
+
     include /etc/nginx/modules-enabled/*.conf;
 
     events {

diff --git a/zammad/templates/deployment-nginx.yaml b/zammad/templates/deployment-nginx.yaml
@@ -0,0 +1,60 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: {{ template "zammad.fullname" . }}-nginx
+  labels:
+    {{- include "zammad.labels" . | nindent 4 }}
+    app.kubernetes.io/component: zammad-nginx
+spec:
+  replicas: {{ .Values.zammadConfig.nginx.replicas }}
+  selector:
+    matchLabels:
+      {{- include "zammad.selectorLabels" . | nindent 6 }}
+  template:
+    metadata:
+      {{- with .Values.podAnnotations }}
+      annotations:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      labels:
+        {{- include "zammad.labels" . | nindent 8 }}
+        app.kubernetes.io/component: zammad-nginx
+    spec:
+      {{- include "zammad.podSpec.deployment" . | nindent 6 }}
+      containers:
+        {{- with .Values.zammadConfig.nginx.sidecars }}
+        {{- toYaml . | nindent 8}}
+        {{- end }}
+        - name: {{ .Chart.Name }}-nginx
+          {{- include "zammad.containerSpec" (merge (dict "containerConfig" .Values.zammadConfig.nginx) .) | nindent 10 }}
+          command:
+            - /usr/sbin/nginx
+            - -g
+            - 'daemon off;'
+          env:
+            {{- include "zammad.env" . | nindent 12 }}
+            {{- include "zammad.env.failOnPendingMigrations" . | nindent 12 }}
+          ports:
+            - name: http
+              containerPort: 8080
+          volumeMounts:
+            {{- include "zammad.volumeMounts" . | nindent 12 }}
+            - name: {{ include "zammad.fullname" . }}-nginx
+              mountPath: /etc/nginx/nginx.conf
+              subPath: nginx.conf
+              readOnly: true
+            - name: {{ include "zammad.fullname" . }}-nginx
+              mountPath: /etc/nginx/sites-enabled/default
+              subPath: default
+              readOnly: true
+            - name: {{ include "zammad.fullname" . }}-tmp
+              mountPath: /var/log/nginx
+      volumes:
+        {{- include "zammad.volumes" . | nindent 8 }}
+        - name: {{ template "zammad.fullname" . }}-init
+          configMap:
+            name: {{ template "zammad.fullname" . }}-init
+            defaultMode: 0755
+        - name: {{ template "zammad.fullname" . }}-nginx
+          configMap:
+            name: {{ template "zammad.fullname" . }}-nginx