improve the rejection logic for wlm

[kaushalmahi12]

Description

In workload management feature the rejection only works for enforced type of query groups. But rejection should work for soft type of query groups as well if the node is in duress. This change enhances the rejection mechanism in wlm.

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 075fb23: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[kaushalmahi12]

Unrelated failure: org.opensearch.remotestore.RemoteStoreUploadIndexPathIT.testRemoteIndexPathFileCreation

[github-actions]

❌ Gradle check result for 075fb23: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[jainankitk]

Unrelated failures:

[Test Result](https://build.ci.opensearch.org/job/gradle-check/49945/testReport/) (5 failures / +3)

    [org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock](https://build.ci.opensearch.org/job/gradle-check/49945/testReport/junit/org.opensearch.cluster/MinimumClusterManagerNodesIT/testThreeNodesNoClusterManagerBlock/)
    [org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock](https://build.ci.opensearch.org/job/gradle-check/49945/testReport/junit/org.opensearch.cluster/MinimumClusterManagerNodesIT/testThreeNodesNoClusterManagerBlock_2/)
    [org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock](https://build.ci.opensearch.org/job/gradle-check/49945/testReport/junit/org.opensearch.cluster/MinimumClusterManagerNodesIT/testThreeNodesNoClusterManagerBlock_3/)
    [org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock](https://build.ci.opensearch.org/job/gradle-check/49945/testReport/junit/org.opensearch.cluster/MinimumClusterManagerNodesIT/testThreeNodesNoClusterManagerBlock_4/)
    [org.opensearch.indices.settings.SearchOnlyReplicaIT.testFailoverWithSearchReplica_WithoutWriterReplicas](https://build.ci.opensearch.org/job/gradle-check/49945/testReport/junit/org.opensearch.indices.settings/SearchOnlyReplicaIT/testFailoverWithSearchReplica_WithoutWriterReplicas/)

[github-actions]

✅ Gradle check result for 3cf64b7: SUCCESS

[codecov]

Codecov Report

Attention: Patch coverage is 33.33333% with 2 lines in your changes missing coverage. Please review.

Project coverage is 72.14%. Comparing base (5120efb) to head (3cf64b7).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...ain/java/org/opensearch/wlm/QueryGroupService.java	33.33%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #16417      +/-   ##
============================================
+ Coverage     72.03%   72.14%   +0.11%     
- Complexity    64998    65071      +73     
============================================
  Files          5313     5313              
  Lines        303293   303294       +1     
  Branches      43886    43888       +2     
============================================
+ Hits         218476   218818     +342     
+ Misses        66863    66545     -318     
+ Partials      17954    17931      -23     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-16417-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 6891267608913dc27b21a65156ddd8817a8e9746
# Push it to GitHub
git push --set-upstream origin backport/backport-16417-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-16417-to-2.x.