Skip to content

Commit

Permalink
Merge pull request #595 from AndreKurait/AddProxyFailureModes
Browse files Browse the repository at this point in the history 
Add Documentation for proxy failure modes
  • Loading branch information
@AndreKurait
AndreKurait authored Apr 23, 2024
2 parents 15c6271 + 987125c commit ccb316d
Showing 1 changed file with 15 additions and 0 deletions.
15 changes: 15 additions & 0 deletions TrafficCapture/trafficCaptureProxyServer/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,18 @@ Follow documentation for [deploying solution](../../deployment/README.md) to set
2. Run the following command to list the Kafka topics, and confirm that the `logging-traffic-topic` has been created.
* For **default/Docker Kafka** clusters `./kafka-tools/kafka/bin/kafka-topics.sh --bootstrap-server "$MIGRATION_KAFKA_BROKER_ENDPOINTS" --list`
* For **AWS MSK(Kafka)** clusters `./kafka-tools/kafka/bin/kafka-topics.sh --bootstrap-server "$MIGRATION_KAFKA_BROKER_ENDPOINTS" --list --command-config kafka-tools/aws/msk-iam-auth.properties`

## Failure Modes

Definitions:
* Mutating - HTTP Requests with method POST, PUT, DELETE, or PATCH with differentiating behavior defined in [ProxyChannelInitializer::shouldGuaranteeMessageOffloading](https://github.com/opensearch-project/opensearch-migrations/blob/15c62718032e02a089e158cdf61c753542c20175/TrafficCapture/trafficCaptureProxyServer/src/main/java/org/opensearch/migrations/trafficcapture/proxyserver/netty/ProxyChannelInitializer.java#L37)

| Component | Type of Request | Scenario | First Behavior | First Duration | Eventual Behavior |
|-----------------|-----------------|-------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Kafka** | Mutating | Kafka Error | Data will be buffered and retried by the Kafka client based on the configured timeouts and the Kafka server's response. The delivery.timeout.ms setting of 10 seconds is the upper limit for how long the client will attempt to send data, including retries, before considering the operation failed. This is in the critical line for request proxying so the client will not receive a response. | 10 Seconds | When the completable future for offloading fails, a log statement will be outputted and the request will be forwarded to the destination service. No retry of this message's offloading will occur |
| **Kafka** | Mutating | Kafka Unavailable | Same behavior as <Kafka Error> | 10 Seconds | Same behavior as <Kafka Error> |
| **Kafka** | Non-Mutating | Kafka Error | Data will be buffered and retried by the Kafka client based on the configured timeouts and the Kafka server's response. The delivery.timeout.ms setting of 10 seconds is the upper limit for how long the client will attempt to send data, including retries, before considering the operation failed. This is not in the critical line for request proxying so request will be immediately proxied. | N/A | N/A |
| **Kafka** | Non-Mutating | Kafka Unavailable | Same behavior as <Kafka Error> | N/A | N/A |
| **Destination** | Any | Offline | Proxy will directly pass through to the client the same behavior/response as seen by the destination. The proxy maintains 1:1 connection ratio between upstream clients and the destination. | N/A | N/A |
| **Logging** | Any | Any | Capture Proxy logs to Standard Output/Error streams. This can be configured via Log4j2 properties which default to non-blocking behavior as specified in [PR#602](https://github.com/opensearch-project/opensearch-migrations/pull/602). For specific AWS Behavior, see configuration for the [AWS Log Driver](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_awslogs.html) | N/A | N/A |
| **Otel** | Any | Any | By default there is no retry of the OTLP Exporter. This can be enabled by setting the appropriate JVM parameter or environment variable as specified in the [Java OTLP Documentation](https://github.com/open-telemetry/opentelemetry-java/tree/main/sdk-extensions/autoconfigure#otlp-exporter-span-metric-and-log-exporters) | N/A | N/A |

0 comments on commit ccb316d

Please sign in to comment.