-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replayer instrumentation #475
Replayer instrumentation #475
Commits on Nov 16, 2023
-
Move addMetricsIfPresent into the metrics builder as a first class me…
…thod for others to leverage. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for a4caca7 - Browse repository at this point
Copy the full SHA a4caca7View commit details
Commits on Nov 27, 2023
-
WIP to play with OpenTelemetry metric instruments and tracer spans.
Most of this is just playing, but making the StreamManager implement AutoCloseable gives a place to end spans to show how long a serializer/connection factory was relevant for. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for c026588 - Browse repository at this point
Copy the full SHA c026588View commit details
Commits on Nov 30, 2023
-
Get gradle files and docker-compose in order to support otlp exports …
…to the collector to prometheus, zipkin, etc Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for f3c0077 - Browse repository at this point
Copy the full SHA f3c0077View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7fb8e2e - Browse repository at this point
Copy the full SHA 7fb8e2eView commit details -
Restore the docker-compose single-node/multi-node split docker-compos…
…e config hierarchy. This was broken from the merge https://github.com/opensearch-project/opensearch-migrations/pull/376/files#diff-430f89dc33402ecf692b9a8372f66e585bb2f9215596433216580efc2a56795c. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for a8ae3d1 - Browse repository at this point
Copy the full SHA a8ae3d1View commit details -
Add labels to each metric instrument so that multiple values can be p…
…lotted within the same graph in prometheus. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for da9d36b - Browse repository at this point
Copy the full SHA da9d36bView commit details -
Move the MetricsClosure into its own class and stop stuffing the metr…
…ics into an optional. Dropping the optionals makes the code simpler and if we don't want to do logging, we can just not fill in the configuration for the SDK. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 06618ca - Browse repository at this point
Copy the full SHA 06618caView commit details -
WIP - Cleanup + get Jaeger to work by switching the endpoint. Also in…
…troduce some more typesafe wrappers for contexts. Lots more to come. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for aba1aab - Browse repository at this point
Copy the full SHA aba1aabView commit details -
Start moving away from ThreadLocal and 'current contexts' and toward …
…explicitly passing strongly typed context objects. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 900bc6d - Browse repository at this point
Copy the full SHA 900bc6dView commit details -
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 3746a8e - Browse repository at this point
Copy the full SHA 3746a8eView commit details -
Configuration menu - View commit details
-
Copy full SHA for e0e7bf1 - Browse repository at this point
Copy the full SHA e0e7bf1View commit details -
Attempt to fix a failing unit test.
Make sure that the context is using the right requestKey, which also will have the appropriate indices as per the test context. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 4b43262 - Browse repository at this point
Copy the full SHA 4b43262View commit details -
Refactor. Couple name changes, class package changes, and moved IRepl…
…ayerRequestContext to the replayer Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 322e12f - Browse repository at this point
Copy the full SHA 322e12fView commit details
Commits on Dec 1, 2023
-
Bundle all of the offloader spans with the netty handler spans.
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 723bf77 - Browse repository at this point
Copy the full SHA 723bf77View commit details
Commits on Dec 2, 2023
-
Improve the tracing story for the capture proxy.
Don't bother showing the Kakfa offloader just buffering (was called recordStream). Now the offloader span is a child span of the connection span from the handler, so we can see the handler gathering the request/response (or waiting for the response). Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 15a1705 - Browse repository at this point
Copy the full SHA 15a1705View commit details -
Tracing change: Flatten the flush span and just record it as 'blocked'.
That makes it a separate state for the logging handler superclass. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 8a6f52a - Browse repository at this point
Copy the full SHA 8a6f52aView commit details
Commits on Dec 4, 2023
-
Minor cleanup - stop setting the namespace or trying to change in a p…
…rocessor. Prometheus metrics already have an export_name that is unique, the processors weren't doing anything useful, & the namespace was appending EVERYTHING from one of the two services. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for c50e01d - Browse repository at this point
Copy the full SHA c50e01dView commit details -
Start instrumenting the replayer with more contexts so that traces an…
…d (less so for now) metrics can be exported across more of the lifetime of a request/connection. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 17c517d - Browse repository at this point
Copy the full SHA 17c517dView commit details
Commits on Dec 11, 2023
-
Double down on using Context objects in lieu of String labels and fix…
… a test bug. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 6288844 - Browse repository at this point
Copy the full SHA 6288844View commit details -
Merge branch 'FixKafkaResume' into OtelMetricsAndTraces
Signed-off-by: Greg Schohn <greg.schohn@gmail.com> # Conflicts: # TrafficCapture/nettyWireLogging/src/main/java/org/opensearch/migrations/trafficcapture/netty/ConditionallyReliableLoggingHttpRequestHandler.java # TrafficCapture/nettyWireLogging/src/main/java/org/opensearch/migrations/trafficcapture/netty/LoggingHttpRequestHandler.java # TrafficCapture/nettyWireLogging/src/test/java/org/opensearch/migrations/trafficcapture/netty/ConditionallyReliableLoggingHttpRequestHandlerTest.java # TrafficCapture/trafficCaptureProxyServer/src/main/java/org/opensearch/migrations/trafficcapture/proxyserver/netty/ProxyChannelInitializer.java # TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/Accumulation.java # TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/CapturedTrafficToHttpTransactionAccumulator.java # TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/RequestResponsePacketPair.java # TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/RequestSenderOrchestrator.java # TrafficCapture/trafficReplayer/src/test/java/org/opensearch/migrations/replay/SimpleCapturedTrafficToHttpTransactionAccumulatorTest.java
Configuration menu - View commit details
-
Copy full SHA for 09e849c - Browse repository at this point
Copy the full SHA 09e849cView commit details
Commits on Dec 12, 2023
-
Merge branch 'main' into OtelMetricsAndTraces
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 9cf2540 - Browse repository at this point
Copy the full SHA 9cf2540View commit details -
Update the Http Logging Handler to suppress response packet captures …
…when the request was ignored and remove the now-unused file for responses. I'd like to revisit this eventually to make sure that it's as efficient as possible and to organize it better. However, it does get the job done for now and tests were updated to confirm. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for c14da6a - Browse repository at this point
Copy the full SHA c14da6aView commit details -
File rename since the LoggingHttpRequest handler now handles both req…
…uests and responses. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 7de4009 - Browse repository at this point
Copy the full SHA 7de4009View commit details
Commits on Dec 15, 2023
-
Shuffling lots of details of Contexts and the relationships between d…
…ifferent levels of them. This has the minimal amount of work to get those relationships to simply compile. Nearly every unit test fails and the code is more clunky than it needs to be, but getting to this point was alone a major lift. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for a5bfc7d - Browse repository at this point
Copy the full SHA a5bfc7dView commit details -
Lots of refactoring to get a couple more test cases to pass.
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 3d60106 - Browse repository at this point
Copy the full SHA 3d60106View commit details -
Configuration menu - View commit details
-
Copy full SHA for ad6bd13 - Browse repository at this point
Copy the full SHA ad6bd13View commit details
Commits on Dec 16, 2023
-
Begin to cleanup endspans for some of the contexts. Lots of bugs rema…
…in, but the replayer isn't crashing. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 07ae016 - Browse repository at this point
Copy the full SHA 07ae016View commit details
Commits on Dec 17, 2023
-
More work to get context chains to work better together.
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for ccb517c - Browse repository at this point
Copy the full SHA ccb517cView commit details -
Two critical bugfixes around handling close observations that were di…
…scovered by trace inspection. 1) close was called AFTER the RRPair was rotated, so there were no traffic streams being committed, resulting in a perpetual hole in the commit log. 2) close was being scheduled immediately, before all requests (in most cases) because the channelInteractionNumber was being miscalculated. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 8bda2e3 - Browse repository at this point
Copy the full SHA 8bda2e3View commit details
Commits on Dec 18, 2023
-
Fix some test code where the nodeId and connectionId got reversed, ca…
…using the same ConnectionReplaySession to be returned for every connection, which resulted in serious corruption of ordering and tests never completing. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for d9df3fa - Browse repository at this point
Copy the full SHA d9df3faView commit details
Commits on Dec 19, 2023
-
Extra guards to try to make tests more reliable, but one of the FullT…
…rafficTest runs (-1, false) is still missing a message. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for ab3dfb4 - Browse repository at this point
Copy the full SHA ab3dfb4View commit details -
More test fixes, including fixing a regression that I had caused in a…
…n earlier edit around flushing the streams held list on close. That should only happen when the close was on an accumulated pair that had a broken request that wasn't rotated. Otherwise, the traffic streams would be committed twice. I also tweaked makeTrafficStreams to keep stable ids on the individual streams to make debugging runs a LOT simpler. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 273c5aa - Browse repository at this point
Copy the full SHA 273c5aaView commit details -
Fix a race condition with commitKafkaKey.
It could be called from one thread while the nextCommit maps were being modified by other (Kafka Consumer) threads. Now there's a lock and a quick copy to protect against that race. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for e0167f5 - Browse repository at this point
Copy the full SHA e0167f5View commit details
Commits on Dec 20, 2023
-
Two changes to kafka interactions. Add trace spans for traffic source…
…/kafka interactions + preempt blocking in the BlockingTrafficSource when there's keys (offsets) to commit. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 3d81ad8 - Browse repository at this point
Copy the full SHA 3d81ad8View commit details -
Extract an IInstrumentationAttributes interface from IScopedInstrumen…
…tationAttributes. This allows for passing root contexts around that have attributes but don't have an associated span. This helps make the code have less assumptions about how it is situated. The change also opens the door to removing all of the static factories for spans and metrics. Those factories can be chained from a top-level context that is passed throughout the callstack, rooted from these new IInstrumentationAttributes classes. That change isn't here, but will probably be completed in the near future & this makes it easier. I'm also in the process of adding contexts and spans to more places (like the high-level traffic source work), which is what precipitated the greater change. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for ae45a6a - Browse repository at this point
Copy the full SHA ae45a6aView commit details
Commits on Dec 21, 2023
-
Checkpoint/WIP - More spans across the board, specifically through th…
…e target transaction. Unfortunately, the TrafficReplayer fails to load because of a race condition around statically initializing the Otel SDK. However, all unit tests are working, so this is a checkpoint release before I remove the reliance on static otel initialization and move toward it being done via contexts. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 195d0ba - Browse repository at this point
Copy the full SHA 195d0baView commit details
Commits on Dec 22, 2023
-
Test bugfix. toString() wasn't threadsafe. Now it is.
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for bef9944 - Browse repository at this point
Copy the full SHA bef9944View commit details -
Refactoring and code consolidation around context management.
The static initializer race conditions have been resolved (no more static otel for tracing and metering) and the E2E solution is functional again. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 8b50c89 - Browse repository at this point
Copy the full SHA 8b50c89View commit details -
Refactoring. Which classes emit metrics, all scopes have names, and g…
…rouping more contexts together in files. 1) RootOtelContext creates all SimpleMeteringClosures and IInstrumentationAttributes has default methods to wire together all metric emissions (making it a lot more convenient). 2) All IInstrumentationAttributes have a scopeName now so that it's consistent and can be used to make metrics and spans. 3) All replayer contexts were split into interfaces and implementations (bridge or PIMPL pattern). I've moved them around so that they now all reside in the tracing directory in the same type of structure. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 0e5fe09 - Browse repository at this point
Copy the full SHA 0e5fe09View commit details
Commits on Dec 23, 2023
-
Refactor TestContext so that it doesn't use statics and allows caller…
…s to bring it up with or w/out InMemory exporters. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 613a504 - Browse repository at this point
Copy the full SHA 613a504View commit details -
Remove a hardcoded path to my local directory
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 7df0dcc - Browse repository at this point
Copy the full SHA 7df0dccView commit details -
Fixed bugs in trace management and forced a lot more test code to tak…
…e a context into it so that span verifications can happen within unit tests. The FullTrafficReplayerTest now has a test that is verifying the number of spans that were reported. That was usefuly to work through to fix the bugs in double counting some spans. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for de0d482 - Browse repository at this point
Copy the full SHA de0d482View commit details -
Add another scheduled span before the request is sent.
I'm not adding one BETWEEN writes yet because that code is too complicated and should be simplified. I'm also not sure that that isn't going to flood the traces for limited value. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 72b1ca8 - Browse repository at this point
Copy the full SHA 72b1ca8View commit details
Commits on Dec 26, 2023
-
Move all span names into virtual interface functions so that they can…
… be used for some metrics too. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 6043eed - Browse repository at this point
Copy the full SHA 6043eedView commit details
Commits on Jan 2, 2024
-
Pass more contexts, make contexts able to express more metrics, and e…
…mit more. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 37b99eb - Browse repository at this point
Copy the full SHA 37b99ebView commit details
Commits on Jan 3, 2024
-
Minor bugfixes that make a huge difference. Fix a broken unit test as…
… a metric name had changed and meter delta events as upDownCounters. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 6684486 - Browse repository at this point
Copy the full SHA 6684486View commit details -
Some refactoring to increase the typesafety and to support greater co…
…ntrol over which attributes are included within metrics and spans. I've also added "activeConnection" for capture connections. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 7bf4388 - Browse repository at this point
Copy the full SHA 7bf4388View commit details
Commits on Jan 4, 2024
-
Fix mend security issue for json-path CVE by updating opensearch-secu…
…rity to 2.11.1.0 Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 8ef0376 - Browse repository at this point
Copy the full SHA 8ef0376View commit details -
Remove zipkin as a tracing sink.
I've personally been using jaeger more and zipkin has been crashing with an OOM on startup (probably due to my docker setup) and I suspect that it has been causing greater problems with the otel collector. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 37ae548 - Browse repository at this point
Copy the full SHA 37ae548View commit details
Commits on Jan 5, 2024
-
Make attribute name filtering more generic and fix a bug in negation …
…so that the connectionId is now emitted from the activeConnections metric. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 1172912 - Browse repository at this point
Copy the full SHA 1172912View commit details
Commits on Jan 8, 2024
-
Minor tweaks to the otel collector (including renaming from 'demo') a…
…nd adding some TODOs for future research. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 22296b7 - Browse repository at this point
Copy the full SHA 22296b7View commit details -
Set the aggregation temporality to delta rather than cumulative.
I still need to continue to knock the dimensionality of the data (unique attributes) down considerably, but this at least mitigates the grpc overflow errors that I was seeing. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for d1b237a - Browse repository at this point
Copy the full SHA d1b237aView commit details -
Wrap all metric emissions within the context's span so that the metri…
…c can be emitted with the span data as its exemplar. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for fdd8141 - Browse repository at this point
Copy the full SHA fdd8141View commit details
Commits on Jan 9, 2024
-
In progress changes. I'm trying to track down a regression and want t…
…o preserve new work first. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 490521d - Browse repository at this point
Copy the full SHA 490521dView commit details
Commits on Jan 10, 2024
-
In-progress checkpoint (code won't compile). Setting up separate metr…
…ic instruments for each of the contexts' needs. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for f199e98 - Browse repository at this point
Copy the full SHA f199e98View commit details -
Another in-progress checkpoint (still won't compile) where I'm moving…
… metric instruments into the context classes. I'm not happy with the scope type propagation through so much of the interface hierarchy, so I'm going to revisit that next. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for e110540 - Browse repository at this point
Copy the full SHA e110540View commit details
Commits on Jan 11, 2024
-
Another checkpoint that still doesn't compile, but less files (I thin…
…k) are problematic. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 156ae72 - Browse repository at this point
Copy the full SHA 156ae72View commit details -
Stop passing the Root telemetry scope as a generic parameter to all o…
…f the instrumentation interfaces. That was only needed for one call, which is easily inlined into the implementation classes by putting createChildContext() calls into each parent. That allows the implementations to go back to their own root scopes and do whatever is necessary. Lots of code is a lot easier to read and maintain now. There are still other changes that I'm in the process of making, including supporting linked spans and squashing compilation errors (still). Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 5dc32d9 - Browse repository at this point
Copy the full SHA 5dc32d9View commit details
Commits on Jan 12, 2024
-
Working on updating proxy code to get everything to compile.
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 320e9d8 - Browse repository at this point
Copy the full SHA 320e9d8View commit details -
More refactoring, still doesn't all compile, but most of it does (top…
…-level Capture proxy is remaining). I need to stitch together a couple root context classes to provide disparate top-level contexts. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 9601a68 - Browse repository at this point
Copy the full SHA 9601a68View commit details
Commits on Jan 13, 2024
-
Fix the last of the compilation errors though tests are failing still.
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 5f6bb3f - Browse repository at this point
Copy the full SHA 5f6bb3fView commit details
Commits on Jan 14, 2024
-
Bugfixes and test fixes to get all of the unit tests to pass.
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for ccc0e2a - Browse repository at this point
Copy the full SHA ccc0e2aView commit details
Commits on Jan 15, 2024
-
Bugfixes. Stop metering double events in a couple spots and fix a con…
…nection id naming bug that was causing FullReplayerTests to fail. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 0744424 - Browse repository at this point
Copy the full SHA 0744424View commit details -
Configuration menu - View commit details
-
Copy full SHA for 82f8ebb - Browse repository at this point
Copy the full SHA 82f8ebbView commit details
Commits on Jan 16, 2024
-
Upgrade otel libraries to 1.34.1 from 1.32 and add the enable_open_me…
…trics flag to the prometheus exporter for the otel collector to support exemplars. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 35a9185 - Browse repository at this point
Copy the full SHA 35a9185View commit details -
Fix a bug where the current scope's attributes weren't being added in…
…to its own span. I've also updated linked spans to allow for more than one (will test shortly) Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 0e8379d - Browse repository at this point
Copy the full SHA 0e8379dView commit details
Commits on Jan 18, 2024
-
Add a TestContext for every replayer test via inheritance on the Test…
… class so that I can initialize and teardown the context (and do some sanity checks in the process). I've already found and fixed a couple issues with scoped contexts being doubly closed, which has resulted in UpDownCounters being corrupted. The final check on the TestContext to make sure that all scopes are closed isn't active yet since the results are "862 tests completed, 808 failed". Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for a076bc3 - Browse repository at this point
Copy the full SHA a076bc3View commit details
Commits on Jan 19, 2024
-
Change how MetricInstruments classes are instantiated.
I had been pulling activity names from the enclosing class, but in cases of inheritance, that was unmaintainable. That had caused a number of automatically-generated metrics to be missing because subclasses would use the superclass ACTIVITY_NAME definition. Now, all the MetricInstruments constructors are private and the objects are created via factory functions that are alongside the inner class definitions. That moves the naming responsibilities to the actual part of the code that cares and makes the system more foolproof. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for d3ee4f1 - Browse repository at this point
Copy the full SHA d3ee4f1View commit details -
Build fix - When refactoring to use TestContexts more globally, a tes…
…t setup method was corrupted. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 54c5e27 - Browse repository at this point
Copy the full SHA 54c5e27View commit details -
Spin up a grafana container in the docker solution with simple creden…
…tials. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 0ce6694 - Browse repository at this point
Copy the full SHA 0ce6694View commit details
Commits on Jan 21, 2024
-
Start to get Source/Target comparison metrics in place and more refac…
…toring to rely upon values within contexts rather than passing redundant copies for things like request/channel keys. Logging comparison metrics as logs is being phased out, being replaced by using metric and span attributes to directly pick up values (and letting those downstream dashboards to add additional logic to determine if values were matching/not, though statusMatch does still exist as a metric - only because its quick and easy on both ends). All tests pass, but I haven't done any testing with the dockerSolution. I need to re-enable the kafka container test because I need to be validating constantly that that test is working, but as of now, it throws an OOM error, which needs to be investigated. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for c8674eb - Browse repository at this point
Copy the full SHA c8674ebView commit details
Commits on Jan 22, 2024
-
Minor cleanup on exception tracking
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for ffcc09b - Browse repository at this point
Copy the full SHA ffcc09bView commit details -
Bugfix - a class was inheriting from the Connection context's MetricI…
…nstruments class when it should not have been Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for dd89336 - Browse repository at this point
Copy the full SHA dd89336View commit details -
Cleanup build.gradle files' open-telemetry dependencies. Embrace otel…
… as an api dependency for coreUtilities, since that's a tight coupling. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for a15d08a - Browse repository at this point
Copy the full SHA a15d08aView commit details
Commits on Jan 23, 2024
-
Partial checkin to delete dead code and clean up imports and style is…
…sues. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 5c454ca - Browse repository at this point
Copy the full SHA 5c454caView commit details -
Addressing PR Feedback with some localized cleanups
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for bf8ea86 - Browse repository at this point
Copy the full SHA bf8ea86View commit details
Commits on Jan 24, 2024
-
aws cli wasn't functional within my arm64 container because the docke…
…rfile was hardcoded for x86_64. Install awscli via pip now, which should work on both environments. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for b641c5a - Browse repository at this point
Copy the full SHA b641c5aView commit details
Commits on Jan 26, 2024
-
Rework otel-collector container packaging.
Support 3 different otel-collector container configurations for docker-compose. * Prometheus + Jaeger + OpenSearch (for metrics, traces, and 'analytics/logs') - all of which are local * AWS CloudWatch + AWS X-Ray (using a mounted credentials file if present, or reliant how well the base image can resolve credentials) + OpenSearch (local) * All of the above `./gradlew :dockerSolution:composeUp` will launch the first configuration by default. Passing the flag `-Potel-collector=otel-aws.yml` will use the AWS configuration and `-Potel-collector=otel-everything.yml` will use the 'everything' configuration. The otel-collector container itself still uses the custom-built collector from mikaylathompson as the base image. However, the extended image normalizes the user and entrypoint so that further extensions and applications of the container can behave as if the base image was the AWS Distro for OpenTelemetry collector base image (amazon/aws-otel-collector, see https://aws-otel.github.io/docs/setup/docker-images). That also includes being able to convert the credentials file into environment variables since the otel-collector was otherwise struggling to use the credentials file directly. The otel-collector configuration is contained within one yaml file and the code within otelConfigs now allows one to fully materialize an otel-collector config file. The otel-config*.yaml files within the dockerSolution have been created via the makeConfigFiles.sh script. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for cd34b32 - Browse repository at this point
Copy the full SHA cd34b32View commit details
Commits on Jan 27, 2024
-
Use System.nanotime() instead of Instant.now() for duration calculations. Call the fillAttributes() method from the super (class or interface) before doing any more puts. Correct the order that attributes are pulled from the scope hierarchy (and add a test) Use the ActivityNames pattern for IWireCaptureContexts StreamLifecycleManager no longer implements Autoclosable. I didn't need it to close down contexts and no classes had implemented a non-empty version except to log that it was called. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for a4173c0 - Browse repository at this point
Copy the full SHA a4173c0View commit details -
Change path from otelcol to otelCollector and enable the collector an…
…d OS analytics engine by default. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 2072f69 - Browse repository at this point
Copy the full SHA 2072f69View commit details
Commits on Jan 28, 2024
-
Split the implementations of fillAttributes into two.
One for attributes that should be present in all sub-spans and another where the attributes should only be present within the current one (and not children) Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for be689b5 - Browse repository at this point
Copy the full SHA be689b5View commit details -
Fix the dependencies for logging leaves and add 'processors' and 'rec…
…eivers' for metrics, traces and logs templates. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 607ff05 - Browse repository at this point
Copy the full SHA 607ff05View commit details -
Setting the docker command for the otel-collector service to use the …
…aws config file Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for d480ed8 - Browse repository at this point
Copy the full SHA d480ed8View commit details -
Set the permissions for the otel container to write to cloudwatch and…
… xray Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 59db42f - Browse repository at this point
Copy the full SHA 59db42fView commit details
Commits on Jan 31, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 76c8c31 - Browse repository at this point
Copy the full SHA 76c8c31View commit details
Commits on Feb 1, 2024
-
Fix the runTestBenchmarks script to work when the endpoint uses http …
…instead of https. I've also removed the 'no-ssl' option and deduce it from the protocol Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 2cc67fd - Browse repository at this point
Copy the full SHA 2cc67fdView commit details -
Merge branch 'main' into ReplayerInstrumentation
Signed-off-by: Greg Schohn <greg.schohn@gmail.com> # Conflicts: # test/awsE2ESolutionSetup.sh
Configuration menu - View commit details
-
Copy full SHA for 6e70d1a - Browse repository at this point
Copy the full SHA 6e70d1aView commit details
Commits on Feb 3, 2024
-
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 9425ab6 - Browse repository at this point
Copy the full SHA 9425ab6View commit details
Commits on Feb 4, 2024
-
IInstrumentationAttributes no longer has scope related functionality.…
… That's been pushed down to its subclass IScopedInstrumentationAttributes. Attributes for spans are now filled when the span is closed rather than when it is created. This gives less leeway to being able to override the value w/out changing the value within the context class, but the context class should be the ground truth, record of value, so this seems like it's the right behavior anyway. With those changes, I did some cleanup on the attribute values that were being tracked for tuple comparison. Now status codes are tracked as metric AND span attributes. That should make it MUCH easier to search metrics for specific patterns that popped up in metrics. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for deb19a4 - Browse repository at this point
Copy the full SHA deb19a4View commit details
Commits on Feb 6, 2024
-
Merge branch 'main' into ReplayerInstrumentation
Signed-off-by: Greg Schohn <greg.schohn@gmail.com> # Conflicts: # TrafficCapture/captureOffloader/src/test/java/org/opensearch/migrations/trafficcapture/StreamChannelConnectionCaptureSerializerTest.java # TrafficCapture/dockerSolution/src/main/docker/migrationConsole/runTestBenchmarks.sh # deployment/cdk/opensearch-service-migration/lib/service-stacks/migration-analytics-stack.ts
Configuration menu - View commit details
-
Copy full SHA for a5f947e - Browse repository at this point
Copy the full SHA a5f947eView commit details -
README documentation for the Instrumentation + some cleanup.
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 9a31210 - Browse repository at this point
Copy the full SHA 9a31210View commit details -
When the first bucket size is <=0 for the CommonScopedMetricInstrumen…
…t constructor override, throw an IllegalArgumentException. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 805e13b - Browse repository at this point
Copy the full SHA 805e13bView commit details
Commits on Feb 7, 2024
-
Add tracing and metrics for replayer sockets as they're created and c…
…losed. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 9aa3432 - Browse repository at this point
Copy the full SHA 9aa3432View commit details -
Bugfix is in the capture proxy's channel context's `sendMeterEventsForEnd()` override to call super so that we'll pickup duration, etc metrics too. The lint fix is in a new python script to facilitate testing from the migration console. The test fix is to give each TrafficReplayer run a fresh TestContext. That context includes channel contexts, which should not be reused across process boundaries and likewise shouldn't be getting reused if we're trying to simulate that for repeated runs. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 48a45ab - Browse repository at this point
Copy the full SHA 48a45abView commit details -
Fix an edge case where a socketChannel might not have been created ev…
…en though the channel context was. Also make minor fixes to other tests to make them more resilient. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 16a9010 - Browse repository at this point
Copy the full SHA 16a9010View commit details
Commits on Feb 8, 2024
-
Handle SocketContexts as first class contexts rather than trying to i…
…mplicitly manage them within a ChannelContext. I've also made a couple more test and production data structures more threadsafe. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 4b102f7 - Browse repository at this point
Copy the full SHA 4b102f7View commit details -
Fix an issue with when to close the SocketContext and some memory lea…
…ks in test code. The SocketContext is now closed in the callback for the channel close rather than before we call close(). That should make a test failure due to duplicate close() calls much less likely and should also give us a better idea of when the socket was actually closed. There were some OutOfMemoryErrors coming back from the github action after 10 minutes of testing. I believe that this was due to having a number of InMemoryMetricExporters that periodic metric exporters were pumping to (in perpetuity, even after the test was complete). We were also potentially tracking lots of backtraces in the ContextTrackers. Both of these now have close() calls that clears all of that logged data. That's now called by TestContext.close(), which is wired for each InstrumentationTest. The next commit will tie off a lot more loose ends, but this commit was tested more extensively, hence the reason that I'm keeping them separate. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 1cb8927 - Browse repository at this point
Copy the full SHA 1cb8927View commit details -
Tie off more loose ends for memory leaks during test runs.
Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 441ca40 - Browse repository at this point
Copy the full SHA 441ca40View commit details
Commits on Feb 9, 2024
-
Test fixes + make scheduled contexts use System.nanotime instead of I…
…nstants, even if an Instant is what's passed in to the constructor Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for dc76239 - Browse repository at this point
Copy the full SHA dc76239View commit details -
Set the x-ray exporter attribute index_all_attributes=true so that at…
…tributes end up as annotations instead of metadata so that they can be searched. To search in x-ray, use `annotationId.ATTRIBUTE_NAME="..."`. Signed-off-by: Greg Schohn <greg.schohn@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 8a875b3 - Browse repository at this point
Copy the full SHA 8a875b3View commit details