Add segment replication + remote store GA information #5037

Naarcha-AWS · 2023-09-15T03:44:16Z

Resolves [DOC] Segment Replication + Remote Storage #4405
Resolves [DOC] Remote Store feature documentation changes for 2.10.0 release #4875
Resolves [DOC]Segment replication added consideration #4984

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

Naarcha-AWS · 2023-09-15T03:48:53Z

@dreamer-89: This PR is ready for your review. Let me know if there is anything I missed.

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

…tion/index.md Co-authored-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

kolchfa-aws

Looks good but we need to reference a consistent name (either remote-backed storage or remote store).

_tuning-your-cluster/availability-and-recovery/remote-store/remote-segment-backpressure.md

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

kolchfa-aws

LGTM

natebower

@Naarcha-AWS Please see my comments and changes and let me know if you have any questions. Thanks!

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

natebower · 2023-09-19T17:15:16Z

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

+As of OpenSearch 2.10, you can use two methods for segment replication:
+
+- With **remote-backed storage**, a persistent storage solution: The primary shard sends segment files to the remote-backed storage and the replica shards source the copy from the same store. For more information about using a remote-backed storage, see [Remote-backed storage]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/remote-store/index/#segment-replication-and-remote-backed-storage).
+- Without remote-backed storage: The primary shard sends segment files directly to the replica shards using node-to-node communication.


Can we call the second method what it is, as opposed to what it isn't? Node-to-node replication?

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Add segment replication + remote store GA information Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add introduction labels Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _tuning-your-cluster/availability-and-recovery/segment-replication/index.md Co-authored-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update index.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Replace Remote Store with Remote Backed Storage Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Suraj Singh <surajrider@gmail.com> Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com>

This reverts commit 8d6bd01.

…ject#5037) * Add segment replication + remote store GA information Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add introduction labels Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _tuning-your-cluster/availability-and-recovery/segment-replication/index.md Co-authored-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update index.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Replace Remote Store with Remote Backed Storage Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Suraj Singh <surajrider@gmail.com> Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com>

* Add segment replication + remote store GA information Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Add introduction labels Signed-off-by: Naarcha-AWS <naarcha@amazon.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _tuning-your-cluster/availability-and-recovery/segment-replication/index.md Co-authored-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update index.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Replace Remote Store with Remote Backed Storage Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Naarcha-AWS <naarcha@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Suraj Singh <surajrider@gmail.com> Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com>

Add segment replication + remote store GA information

96dc030

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

Naarcha-AWS requested review from cwillum, hdhalter, kolchfa-aws, vagimeli, ananzh, seanneumann, AMoo-Miki and natebower as code owners September 15, 2023 03:44

Add introduction labels

779369c

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

anasalkouz reviewed Sep 15, 2023

View reviewed changes

hdhalter assigned Naarcha-AWS Sep 15, 2023

hdhalter added v2.10.0 release-notes PR: Include this PR in the automated release notes 3 - Tech review PR: Tech review in progress labels Sep 15, 2023

Naarcha-AWS commented Sep 15, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS commented Sep 15, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS commented Sep 15, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS commented Sep 15, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS added 2 commits September 15, 2023 12:27

Apply suggestions from code review

591ba8f

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

Apply suggestions from code review

b34a484

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

dreamer-89 suggested changes Sep 15, 2023

View reviewed changes

Update _tuning-your-cluster/availability-and-recovery/segment-replica…

9b668fb

…tion/index.md Co-authored-by: Suraj Singh <surajrider@gmail.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

Bukhtawar reviewed Sep 16, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Bukhtawar reviewed Sep 16, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS commented Sep 18, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS commented Sep 18, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Apply suggestions from code review

c5291f7

Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

Naarcha-AWS requested a review from Bukhtawar September 18, 2023 17:16

Naarcha-AWS requested review from dreamer-89 and anasalkouz September 18, 2023 17:16

Naarcha-AWS commented Sep 18, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS commented Sep 18, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS added 2 commits September 18, 2023 12:28

Apply suggestions from code review

92133af

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

Update index.md

a2e3323

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

kolchfa-aws reviewed Sep 19, 2023

View reviewed changes

Apply suggestions from code review

0fc62f0

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

Naarcha-AWS requested a review from kolchfa-aws September 19, 2023 15:35

Naarcha-AWS added 4 - Doc review PR: Doc review in progress and removed 3 - Tech review PR: Tech review in progress labels Sep 19, 2023

Replace Remote Store with Remote Backed Storage

f4c8df8

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

kolchfa-aws approved these changes Sep 19, 2023

View reviewed changes

Naarcha-AWS added 5 - Editorial review PR: Editorial review in progress and removed 4 - Doc review PR: Doc review in progress labels Sep 19, 2023

natebower reviewed Sep 19, 2023

View reviewed changes

Apply suggestions from code review

1b215d2

Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

Naarcha-AWS merged commit 64e120a into main Sep 19, 2023
4 checks passed

vagimeli added a commit that referenced this pull request Sep 19, 2023

Revert "Add segment replication + remote store GA information (#5037)"

c71fb7a

This reverts commit 8d6bd01.

dreamer-89 mentioned this pull request Sep 25, 2023

[DOC] Update segment replication consideration section #5102

Closed

4 tasks

Naarcha-AWS deleted the segment-replication-ga branch March 28, 2024 23:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add segment replication + remote store GA information #5037

Add segment replication + remote store GA information #5037

Naarcha-AWS commented Sep 15, 2023

Naarcha-AWS commented Sep 15, 2023

kolchfa-aws left a comment

kolchfa-aws left a comment

natebower left a comment

natebower Sep 19, 2023

Add segment replication + remote store GA information #5037

Add segment replication + remote store GA information #5037

Conversation

Naarcha-AWS commented Sep 15, 2023

Checklist

Naarcha-AWS commented Sep 15, 2023

kolchfa-aws left a comment

Choose a reason for hiding this comment

kolchfa-aws left a comment

Choose a reason for hiding this comment

natebower left a comment

Choose a reason for hiding this comment

natebower Sep 19, 2023

Choose a reason for hiding this comment