Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remote Store: GA in 2.10 release #9826

Closed
sachinpkale opened this issue Sep 6, 2023 · 0 comments
Closed

Remote Store: GA in 2.10 release #9826

sachinpkale opened this issue Sep 6, 2023 · 0 comments
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework v2.10.0

Comments

@sachinpkale
Copy link
Member

sachinpkale commented Sep 6, 2023

Overview

Remote store feature was introduced in OpenSearch 2.3 under experimental feature flag. The goal of the feature, as mentioned in the feature proposal (#1968), is to improve durability of the ingested data. It is ready to go GA in 2.10 release. Between 2.3 and 2.10, we added a bunch of incremental functionalities to this feature, added more unit and integration tests, ran performance tests. Following sections provides checklist to take the feature to GA.

Release Meta: #9805

Checklist

Configuration

  1. Remote store feature is built on top of existing repository plugin interface. In order to enable the feature, repository plugin needs to be installed on the cluster.
  2. The feature can be enabled at a cluster level by providing repository details. Once a cluster is remote store enabled, all the indices created in the cluster are remote store enabled.
  3. For remote store enabled index, replication type will always be selected as Segment Replication

Functionality

  1. Request level durability of the indexed data
  2. Remote store integration in Replication, Failover and Peer Recovery flow
  3. Interoperability of remote store with snapshot
  4. Store and Restore index metadata from remote store

Meta Issues

  1. [Remote Store] Experimental Release Tracking #3974
  2. [Remote Store][META] Request Level Durability for for remote-backed indexes (Experimental Release Tracking for 2.5) #5671
  3. [Meta] Remote Store: 2.8.0 - Release Tracking #7382
  4. [Meta] Remote Store: 2.10.0 - Release Tracking #9805
  5. [META] Remote Cluster State Issues #9344

Testing

  1. Unit tests and Integ tests are added as part of each PR with required code coverage
  2. Ran existing integ tests by enabling the remote store settings - As this is quite intrusive change to enable remote store for all the existing tests, the changes were done in private fork and carried out tests with https://github.com/sachinpkale/OpenSearch/tree/remote-store-enabled-its

Performance

Documentation

@sachinpkale sachinpkale added enhancement Enhancement or improvement to existing feature or request untriaged Storage:Durability Issues and PRs related to the durability framework v2.10.0 and removed untriaged labels Sep 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework v2.10.0
Projects
None yet
Development

No branches or pull requests

1 participant