Warn about index.mapper.dynamic usage instead of erroring

[peternied]

Description

When migrating from older versions of Elasticsearch/OpenSearch index.mapper.dynamic could be present, use the deprecation system to warn about this state and ignore this legacy setting so the index is usable.

Related Issues

• Resolves [BUG]For ES version after 6.0.0 the index seems to be created with " index.mapper.dynamic " attribute which ideally it should not have #3879
• Related [BUG] IndexSettings gets into invalid states on a cluster #12350

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)
• Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[peternied]

@shwetathareja Could you take a look into this issue? I'd would also like to backport this change to 2.12 and 1.3

[github-actions]

❕ Gradle check result for 747f7c5: UNSTABLE

• TEST FAILURES:

      3 org.opensearch.cluster.coordination.AwarenessAttributeDecommissionIT.testConcurrentDecommissionAction

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[github-actions]

❌ Gradle check result for 8dd8569: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[reta]

@peternied the DCO check is failing ... :(

[github-actions]

❌ Gradle check result for 549e789: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[peternied]

This because of the version bump is midflight?

=== Standard output of node `node{:qa:mixed-cluster:v2.13.0-0}` ===

»    ↓ errors and warnings from /var/jenkins/workspace/gradle-check/search/qa/mixed-cluster/build/testclusters/v2.13.0-0/logs/opensearch.stdout.log ↓
» WARN ][o.o.g.DanglingIndicesState] [v2.13.0-0] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
» WARN ][o.o.d.FileBasedSeedHostsProvider] [v2.13.0-0] expected, but did not find, a dynamic hosts list at [/var/jenkins/workspace/gradle-check/search/qa/mixed-cluster/build/testclusters/v2.13.0-0/config/unicast_hosts.txt]
»   ↑ repeated 4 times ↑
» ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [v2.13.0-0] uncaught exception in thread [main]
»  org.opensearch.bootstrap.StartupException: OpenSearchException[failed to bind service]; nested: IndexFormatTooNewException[Format version is not supported (resource BufferedChecksumIndexInput(NIOFSIndexInput(path="/var/jenkins/workspace/gradle-check/search/qa/mixed-cluster/build/testclusters/v2.13.0-0/data/nodes/0/_state/_5.cfs") [slice=_5.fnm])): 1 (needs to be between 0 and 0)];
»  	at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:185) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»  	at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:172) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»  	at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»  	at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138) ~[opensearch-cli-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»  	at org.opensearch.cli.Command.main(Command.java:101) ~[opensearch-cli-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»  	at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:138) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»  	at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:104) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»  Caused by: org.opensearch.OpenSearchException: failed to bind service
»  	at org.opensearch.node.Node.<init>(Node.java:1331) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»  	at org.opensearch.node.Node.<init>(Node.java:413) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»  	at org.opensearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:242) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»  	at org.opensearch.bootstrap.Bootstrap.setup(Bootstrap.java:242) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»  	at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:404) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»  	at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:181) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
»  	... 6 more

[reta]

This because of the version bump is midflight?

I think should be fixed, we've just merged #12394 to main

[github-actions]

❌ Gradle check result for 549e789: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 549e789: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❕ Gradle check result for 7e28535: UNSTABLE

• TEST FAILURES:

      1 org.opensearch.index.IndexServiceTests.testAsyncTranslogTrimTaskOnClosedIndex

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-11193-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 f34ca960149c4f37f18bddfa4a0b0d7d2dcee9e4
# Push it to GitHub
git push --set-upstream origin backport/backport-11193-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-11193-to-2.x.

[reta]

@peternied it seems like this fix is incomplete, nonetheless the value of INDEX_MAPPER_DYNAMIC_SETTING should be ignored, it won't be, org.opensearch.action.support.AutoCreateIndex:

    public AutoCreateIndex(
        Settings settings,
        ClusterSettings clusterSettings,
        IndexNameExpressionResolver resolver,
        SystemIndices systemIndices
    ) {
        this.resolver = resolver;
        dynamicMappingDisabled = !MapperService.INDEX_MAPPER_DYNAMIC_SETTING.get(settings);
        this.systemIndices = systemIndices;
        this.autoCreate = AUTO_CREATE_INDEX_SETTING.get(settings);
        clusterSettings.addSettingsUpdateConsumer(AUTO_CREATE_INDEX_SETTING, this::setAutoCreate);
    }