Skip to content

fix: exclude extraSecrets from KbsConfig on Azure to match pull-secret-credential-policy - #42

Open
butler54 wants to merge 1 commit into
validatedpatterns:mainfrom
butler54:fix/azure-kbs-credential-secret
Open

fix: exclude extraSecrets from KbsConfig on Azure to match pull-secret-credential-policy#42
butler54 wants to merge 1 commit into
validatedpatterns:mainfrom
butler54:fix/azure-kbs-credential-secret

Conversation

@butler54

@butler54 butler54 commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Problem

On Azure, the KBS deployment fails with:

Error in creating/updating KBS deployment {"kbsconfig": "trustee-operator-system", "err": "Secret \"credential\" not found"}

Root cause: templates/kbs.yaml unconditionally lists kbs.extraSecrets
(default: ["credential"]) in KbsConfig.spec.kbsSecretResources. But
templates/pull-secret-credential-policy.yaml — the ACM
ConfigurationPolicy that creates the credential Secret from the
cluster's openshift-config/pull-secret — is explicitly (and correctly)
skipped on Azure, since peer-pod CDH doesn't fetch registry credentials via
KBS. Nothing kept KbsConfig's secret list in sync with that exclusion.

Fix

Gate the extraSecrets range in templates/kbs.yaml with the same
condition used by pull-secret-credential-policy.yaml:

{{- if and (eq .Values.global.coco.secured true) (ne .Values.global.clusterPlatform "Azure") }}
{{- range .Values.kbs.extraSecrets }}
- "{{ . }}"
{{- end }}
{{- end }}

Bare metal and other non-Azure platforms are unaffected — extraSecrets
(credential) is still included there, matching existing behavior.

(Note: an earlier revision of this PR took the opposite approach — making
the credential policy unconditional so the secret would exist everywhere.
That was reverted after confirming on a live Azure cluster that
sandboxed-policies-chart's pull-secret-distribution mechanism
(per-namespace pull-secret Secret + default ServiceAccount
imagePullSecrets patch) is already deployed and working there, making the
KBS-based credential resource genuinely redundant on Azure, not just
theoretically unused.)

Verification

helm template / helm lint against a clean checkout of this branch:

  • global.clusterPlatform=Azure, global.coco.secured=truekbsSecretResources excludes "credential"; pull-secret-credential-policy does not render.
  • global.clusterPlatform=BareMetal, global.coco.secured=truekbsSecretResources still includes "credential" (unchanged); policy still renders.
  • helm lint . passes.

Confirmed on a live Azure cluster (sandbox-policies ArgoCD app Synced/Healthy) that pull-secret Secrets and patched default ServiceAccounts already exist in workload namespaces (hello-openshift, kbs-access), independent of KBS.

@butler54
butler54 force-pushed the fix/azure-kbs-credential-secret branch from 6813f90 to f716adf Compare August 31, 2026 14:05
@butler54 butler54 changed the title fix: exclude extraSecrets from KbsConfig on Azure to match pull-secret-credential-policy fix: make pull-secret-credential-policy unconditional across platforms Aug 31, 2026
…t-credential-policy

KbsConfig.spec.kbsSecretResources unconditionally listed kbs.extraSecrets
(default: ["credential"]), but the ACM ConfigurationPolicy that creates the
'credential' Secret (pull-secret-credential-policy.yaml) is explicitly
skipped on Azure, since peer-pod CDH doesn't fetch registry credentials from
KBS there. The mismatch caused the kbsconfig-controller to fail deploying
KBS on Azure with:

  Error in creating/updating KBS deployment: Secret "credential" not found

Gate the extraSecrets range in templates/kbs.yaml with the same
'global.coco.secured == true AND global.clusterPlatform != Azure' condition
used by pull-secret-credential-policy.yaml, so the CR and the policy that
satisfies it stay in sync by construction. Bare metal / other platforms are
unaffected (extraSecrets still included there).

Confirmed on a live Azure cluster that sandboxed-policies-chart's
pull-secret-distribution mechanism (per-namespace 'pull-secret' Secret +
default ServiceAccount imagePullSecrets patch) is already deployed and
working, so the KBS-based 'credential' resource is genuinely unnecessary on
Azure -- not just unused in theory.
@butler54
butler54 force-pushed the fix/azure-kbs-credential-secret branch from f716adf to bac9ef0 Compare August 31, 2026 15:13
@butler54 butler54 changed the title fix: make pull-secret-credential-policy unconditional across platforms fix: exclude extraSecrets from KbsConfig on Azure to match pull-secret-credential-policy Aug 31, 2026
butler54 added a commit to validatedpatterns/coco-pattern that referenced this pull request Aug 31, 2026
On Azure, KBS deployment fails with:
  Error in creating/updating KBS deployment: Secret "credential" not found

trustee-chart's default kbs.extraSecrets (['credential']) is unconditionally
added to KbsConfig.spec.kbsSecretResources, but the ACM ConfigurationPolicy
that creates that Secret (pull-secret-credential-policy.yaml) is skipped on
Azure -- peer-pod CDH doesn't fetch registry credentials from KBS there.

Override kbs.extraSecrets: [] in overrides/values-trustee-azure.yaml so
KbsConfig on Azure never lists a Secret that will never exist.

Confirmed on a live Azure cluster that sandboxed-policies-chart's
pull-secret-distribution mechanism (per-namespace 'pull-secret' Secret +
default ServiceAccount imagePullSecrets patch) is already deployed and
working -- the KBS-based credential path is genuinely redundant on Azure,
not just theoretically unused.

This is a workaround for the currently-pinned trustee chartVersion (0.10.*).
The proper fix (gate extraSecrets in kbs.yaml the same way as the policy) is
in validatedpatterns/trustee-chart#42; once merged and released as >= 0.10.1,
this override becomes redundant but harmless and can be dropped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant