Skip to content

Support configuration overrides for troubleshooting - #2810

Merged
bert-e merged 3 commits into
development/9.6from
improvement/BB-809
Sep 1, 2026
Merged

Support configuration overrides for troubleshooting#2810
bert-e merged 3 commits into
development/9.6from
improvement/BB-809

Conversation

@francoisferrand

@francoisferrand francoisferrand commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Any configuration field should be changeable per process, without a new image or release, so that support can adjust a setting on a running platform. The named settings only cover the knobs a schema field exists for, and cannot reach objects with unconstrained keys, such as the librdkafka producer parameters.

BACKBEAT_CONFIG_OVERRIDES now holds a JSON document applied to the configuration as a JSON Merge Patch, before validation: the merged result is validated as a whole, so a typo or a wrong type fails at startup rather than leaving the setting silently ignored. Each schema is given the fraction of the patch covering its own fields, applied after the environment variables derived from it, so that nothing silently overrides the escape hatch someone reached for precisely because the usual path did not work.

It is applied over the configuration file and any other setting, so that nothing silently overrides the escape hatch someone reached for precisely because the usual path did not work.

This stays an escape hatch: the named settings remain the supported way to configure backbeat.

Issue: BB-809

Comment thread lib/config/configOverrides.js
@codecov

codecov Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.10%. Comparing base (d054ab6) to head (4b0d390).

Additional details and impacted files

Impacted file tree graph

Files with missing lines Coverage Δ
lib/Config.js 81.81% <100.00%> (+0.67%) ⬆️
lib/config/configOverrides.js 100.00% <100.00%> (ø)
lib/config/envOverrides.js 98.70% <100.00%> (+0.01%) ⬆️

... and 2 files with indirect coverage changes

Components Coverage Δ
Bucket Notification 80.25% <ø> (ø)
Core Library 82.36% <100.00%> (+0.04%) ⬆️
Ingestion 71.37% <ø> (ø)
Lifecycle 80.45% <ø> (ø)
Oplog Populator 85.80% <ø> (ø)
Replication 62.13% <ø> (ø)
Bucket Scanner 85.76% <ø> (ø)
@@                 Coverage Diff                 @@
##           development/9.6    #2810      +/-   ##
===================================================
+ Coverage            76.07%   76.10%   +0.03%     
===================================================
  Files                  203      204       +1     
  Lines                14024    14051      +27     
===================================================
+ Hits                 10669    10694      +25     
- Misses                3345     3347       +2     
  Partials                10       10              
Flag Coverage Δ
api:retry 9.46% <29.03%> (+0.01%) ⬆️
api:routes 9.22% <29.03%> (+0.01%) ⬆️
bucket-scanner 85.76% <ø> (ø)
ft_test:queuepopulator 10.84% <29.03%> (-0.62%) ⬇️
ingestion 12.59% <29.03%> (+0.01%) ⬆️
lib 9.21% <29.03%> (+0.01%) ⬆️
lifecycle 19.61% <29.03%> (-0.01%) ⬇️
notification 1.01% <0.00%> (-0.01%) ⬇️
oplogPopulator 0.13% <0.00%> (-0.01%) ⬇️
replication 19.16% <29.03%> (-0.01%) ⬇️
unit 55.57% <100.00%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread docs/config-overrides.md
Comment thread tests/unit/lib/config/Config.spec.js

@SylvainSenechal SylvainSenechal left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • I think this will need a small artesca ticket to document it ? Unless we only want CS to use it and not clients directly 🤔
  • We may wanna exercise config overrides in zenko functional tests. No need to create new specific tests, but for example we have a variable timeFactor to turn one day into a second that maybe we could use it on ?

Comment thread lib/config/configOverrides.js
Comment thread lib/config/configOverrides.js
Base automatically changed from improvement/BB-808 to development/9.6 August 27, 2026 13:13
@bert-e

bert-e commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Hello francoisferrand,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval
/bypass_build_status Bypass the build and test status
/bypass_commit_size Bypass the check on the size of the changeset TBA
/bypass_incompatible_branch Bypass the check on the source branch prefix
/bypass_jira_check Bypass the Jira issue check
/bypass_peer_approval Bypass the pull request peers' approval
/bypass_leader_approval Bypass the pull request leaders' approval
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request.
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@bert-e

bert-e commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

Any configuration field should be changeable per process, without a new
image or release, so that support can adjust a setting on a running
platform. The named settings only cover the knobs a schema field exists
for, and cannot reach objects with unconstrained keys, such as the
librdkafka producer parameters.

BACKBEAT_CONFIG_OVERRIDES now holds a JSON document applied to the
configuration as a JSON Merge Patch, before validation: the merged
result is validated as a whole, so a typo or a wrong type fails at
startup rather than leaving the setting silently ignored. Each
schema is given the fraction of the patch covering its own fields,
applied after the environment variables derived from it, so that
nothing silently overrides the escape hatch someone reached for
precisely because the usual path did not work.

It is applied over the configuration file and any other setting, so
that nothing silently overrides the escape hatch someone reached for
precisely because the usual path did not work.

This stays an escape hatch: the named settings remain the supported
way to configure backbeat.

Issue: BB-809
JSON.parse sets a `__proto__` key as a plain member, so a
`BACKBEAT_CONFIG_OVERRIDES` document naming one had it merged into
Object.prototype: the configuration field the operator meant to set
stayed untouched, and the override silently corrupted every object in
the process instead.

No configuration field is named that, and a JS object cannot hold such a
member anyway, so the key is now dropped when the document is parsed,
and ignored by the merge whatever its caller passes. Reaching the escape
hatch takes operator access, but neither the parsing nor the merge
should depend on that to stay harmless.

Issue: BB-809
Code could crash instead of throwing a "clean" error if transitions are
enabled in config but there is replication is not configured at all.

Issue: BB-809
@francoisferrand

Copy link
Copy Markdown
Contributor Author

/approve

@bert-e

bert-e commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

  • ✔️ development/9.6

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.6
  • development/9.0
  • development/9.1
  • development/9.2
  • development/9.3
  • development/9.4
  • development/9.5

This pull request does not target the following hotfix branch(es) so they
will be left untouched:

  • hotfix/7.9.0
  • hotfix/8.2.12
  • hotfix/7.4.10
  • hotfix/9.0.4
  • hotfix/7.70.12
  • hotfix/7.8.0
  • hotfix/7.10.0
  • hotfix/7.10.3
  • hotfix/9.0.7
  • hotfix/7.4.0
  • hotfix/7.4.8
  • hotfix/7.4.1
  • hotfix/7.4.2
  • hotfix/7.2.0
  • hotfix/7.6.0
  • hotfix/7.10.12
  • hotfix/7.10.2
  • hotfix/7.10.4
  • hotfix/7.70.15
  • hotfix/7.70.1
  • hotfix/7.10.17
  • hotfix/7.10.1
  • hotfix/7.4.6
  • hotfix/7.7.0
  • hotfix/7.4.5
  • hotfix/7.4.7
  • hotfix/7.4.3
  • hotfix/7.4.9
  • hotfix/7.10.8
  • hotfix/7.4.4

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

  • Any commit you add on the source branch will trigger a new cycle after the
    current queue is merged.
  • Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

@bert-e

bert-e commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Queue build failed

The corresponding build for the queue failed:

  • Checkout the status page.
  • Identify the failing build and review the logs.
  • If no issue is found, re-run the build.
  • If an issue is identified, checkout the steps below to remove
    the pull request from the queue for further analysis and maybe rebase/merge.
Remove the pull request from the queue
  • Add a /wait comment on this pull request.
  • Click on login on the status page.
  • Go into the manage page.
  • Find the option called Rebuild the queue and click on it.
    Bert-E will loop again on all pull requests to put the valid ones
    in the queue again, while skipping the one with the /wait comment.
  • Wait for the new queue to merge, then merge/rebase your pull request
    with the latest changes to then work on a proper fix.
  • Once the issue is fixed, delete the /wait comment and
    follow the usual process to merge the pull request.

@bert-e

bert-e commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

I have successfully merged the changeset of this pull request
into targetted development branches:

  • ✔️ development/9.6

The following branches have NOT changed:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.6
  • development/9.0
  • development/9.1
  • development/9.2
  • development/9.3
  • development/9.4
  • development/9.5

Please check the status of the associated issue BB-809.

Goodbye francoisferrand.

@bert-e
bert-e merged commit dfee42b into development/9.6 Sep 1, 2026
24 checks passed
@bert-e
bert-e deleted the improvement/BB-809 branch September 1, 2026 06:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants