Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding cache eviction and listener for invalidating index field type … #142

Merged
merged 7 commits into from
Oct 10, 2024

Conversation

jainankitk
Copy link
Collaborator

…mappings on index deletion/update

Description

This change primarily achieves 2 things:

  • Introduces cache eviction logic for ensuring that the field data cache stays bounded
  • Adds cluster state listener for invalidating index field type mappings on index deletion/update

Issues Resolved

List any issues this PR will resolve, e.g. Closes [...].

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…mappings on index deletion/update

Signed-off-by: Ankit Jain <akjain@amazon.com>
Signed-off-by: Ankit Jain <akjain@amazon.com>
Signed-off-by: Ankit Jain <akjain@amazon.com>
Signed-off-by: Ankit Jain <akjain@amazon.com>
@deshsidd
Copy link
Collaborator

Existing tests still failing. Please take a look @jainankitk

@deshsidd
Copy link
Collaborator

Looks like one of the reasons of the build failures is the license headers:

X:/src/main/java/org/opensearch/plugin/insights/core/service/categorizer/IndicesFieldTypeCache.java

Copy link
Member

@ansjcy ansjcy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left 1 minor comment.
@jainankitk do you think we should add the cache size as part of the healthcheck API as well?
Also we need to add the license header as caught by the build.

@deshsidd
Copy link
Collaborator

if its quick can add unit tests to make sure cache is utilized and evicting properly?

@deshsidd
Copy link
Collaborator

deshsidd commented Oct 10, 2024

LGTM, left 1 minor comment. @jainankitk do you think we should add the cache size as part of the healthcheck API as well? Also we need to add the license header as caught by the build.

would be good to have this as part of the health check APIs in case we see performance issues and need to debug @ansjcy

@deshsidd
Copy link
Collaborator

Thanks @jainankitk, LGTM overall. Only pending items seem to be:

  1. Build fix
  2. Removing the TODO and use the settings
  3. Unit tests?

Signed-off-by: Ankit Jain <akjain@amazon.com>
Signed-off-by: Ankit Jain <akjain@amazon.com>
Signed-off-by: Ankit Jain <akjain@amazon.com>
@deshsidd deshsidd merged commit fb24118 into opensearch-project:main Oct 10, 2024
16 checks passed
@jainankitk jainankitk deleted the ft-cache branch October 10, 2024 20:05
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 10, 2024
#142)

* Adding cache eviction and listener for invalidating index field type mappings on index deletion/update

Signed-off-by: Ankit Jain <akjain@amazon.com>

* Fixing spotless violations

Signed-off-by: Ankit Jain <akjain@amazon.com>

* Fixing code to use .index instead of .findMappings

Signed-off-by: Ankit Jain <akjain@amazon.com>

* Fixing spotless violations

Signed-off-by: Ankit Jain <akjain@amazon.com>

* Addressing review comments

Signed-off-by: Ankit Jain <akjain@amazon.com>

* Fixing existing test failures

Signed-off-by: Ankit Jain <akjain@amazon.com>

* Fixing existing test failures

Signed-off-by: Ankit Jain <akjain@amazon.com>

---------

Signed-off-by: Ankit Jain <akjain@amazon.com>
(cherry picked from commit fb24118)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
jainankitk pushed a commit that referenced this pull request Oct 10, 2024
#142) (#143)

* Adding cache eviction and listener for invalidating index field type mappings on index deletion/update



* Fixing spotless violations



* Fixing code to use .index instead of .findMappings



* Fixing spotless violations



* Addressing review comments



* Fixing existing test failures



* Fixing existing test failures



---------


(cherry picked from commit fb24118)

Signed-off-by: Ankit Jain <akjain@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants