-
Notifications
You must be signed in to change notification settings - Fork 480
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DOC] Create documentation for Remote Cluster State #5053
Comments
Shouldn't this be under https://opensearch.org/docs/latest/tuning-your-cluster/availability-and-recovery/remote-store/index/? |
@sachinpkale It could be under https://opensearch.org/docs/latest/tuning-your-cluster/availability-and-recovery/remote-store/index/ but the content at the main index page is very specific to segment and translog storage. We could create a section for remote cluster state in the index page. |
@soosinha Sorry for the confusion. I was suggesting to have sub-page under remote-store instead of availability-and-recovery. |
@sachinpkale yes i agree we should have sub page under remote-store as remote cluster state would contain metadata beyond Index Metadata as well. |
@Naarcha-AWS Is there a PR corresponding to this issue? Wondering if we have mistakenly closed this issue. |
@ashking94: I'll link the PR in here once its ready. |
@Naarcha-AWS - is this targeting the 2.12 release? |
@hdhalter: This is something we missed in 2.11. Adding a PR for it shortly. |
What do you want to do?
Tell us about your request. Provide a summary of the request and all versions that are affected.
Remote Cluster State
[New sub page under https://opensearch.org/docs/latest/tuning-your-cluster/availability-and-recovery/remote-store]
Remote Cluster State protects against any cluster state metadata loss resulting due to quorum loss (permanently losing majority of cluster manager nodes) in the cluster.
Cluster State in an internal structure which contains the metadata of the cluster along with other information. The metadata includes details about index metadata like settings, mappings, active copies of the shards, cluster level settings, aliases, templates and data streams etc. This metadata is managed by the elected cluster manager node and is essential for proper functioning of the cluster. When the cluster loses majority of the cluster manager nodes permanently, lets say 2 out of 3 cluster manager nodes are lost, then the cluster can experience data loss as there are no guarantees that latest cluster state metadata is present in the surviving nodes. Today, cluster durability is the function of the cluster manager node storage. And, persisting the state to remote provides better durability guarantees.
When remote cluster state feature is enabled, the cluster metadata will be published to a remote repository configured in the cluster. Note that, currently only index metadata will be persisted to remote store in OpenSearch 2.10.
Any time, the new cluster manager nodes are launched after disaster recovery, they will bootstrap using the index metadata stored in the remote repository automatically. Consequently, the data of the indices will also be restored when remote store is enabled.
How to use ?
Add the enabled flag and the repository settings specify below in the yml and start the cluster.
Setting:
cluster.remote_store.state.enabled
Data type: boolean
Properties: Final, NodeScope
This settings controls the enabling of remote cluster state.
Setting:
node.attr.remote_store.state.repository
Data type: String
Properties Node attribute
This settings specifies the repository to be used for remote cluster state storage. The actual repository settings are specifies with the prefix as node.attr.remote_store.repository.<repository_name>.*
Both the above settings should be present in order for remote cluster state to work.
Limitations
What other resources are available? Provide links to related issues, POCs, steps for testing, etc.
The text was updated successfully, but these errors were encountered: