Enhance logging with Async and high performance appenders

[AndreKurait]

Description

Move log4j2 appenders to high performance async loggers:

• Existing logging was performed synchronous and blocking based on appender behavior. Moves this logic to be async with higher performance appender configuration.
• Note: This will still eventually block on flushing out the appenders, but there will be a buffer between logging and io streams.
• Existing Data Flow (All Blocking Calls):
  Application → Log4j2 Appender → Standard Out → ECSLogDriver → Cloudwatch Logs
• New Data Flow (Blocking unless specified):
  Application → AsyncLogger (async)→ Log4j2 Appender → ECSLogDriver → Cloudwatch Logs

The AsyncLogger internally buffers so the rest can be "blocking" and still not block the application flow until it's internal queue is full

Enhance logging with microsecond clock and enhanced performance options
* Added precision on logging and moved console appender to direct reference instead of through standard output stream.

Add multilinePattern to log driver

Fixed some logging unreliable unit tests with specifying a synchronous logger.

Documentation on Proxy Behavior added in PR#595

• Category: Enhancement/Bug Fix
• Why these changes are required?
• What is the old behavior before changes and new behavior after changes? In AWS, logs were one line per line, now combined (e.g. stack trace together). Now microsecond instead of millisecond timing. Higher throughput replayer when debug/trace logging is enabled.

Issues Resolved

MIGRATIONS-1663

Is this a backport? If so, please add backport PR # and/or commits #

Testing

Deployed to AWS and tested in high performance scenarios with high logging (Debug/Trace). Verified multiline behavior in AWS.

Check List

• New functionality includes testing
  • All tests pass, including unit test, integration test and doctest
• New functionality has been documented
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.94%. Comparing base (987125c) to head (643e8e9).
Report is 1 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #602      +/-   ##
============================================
- Coverage     76.02%   75.94%   -0.08%     
  Complexity     1494     1494              
============================================
  Files           162      162              
  Lines          6348     6348              
  Branches        572      572              
============================================
- Hits           4826     4821       -5     
- Misses         1145     1149       +4     
- Partials        377      378       +1     

Flag	Coverage Δ
unittests	75.94% <ø> (-0.08%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mikaylathompson]

Can you elaborate on including this/why it's important?

https://logging.apache.org/log4j/2.x/manual/async.html#location-location-location

The performance impact of taking a stack trace snapshot is even higher for asynchronous loggers: logging with location is 30-100 times slower than without location.

[AndreKurait]

Removed, had originally included it due to the %c{1} in our layout pattern, though reading now that %c means logger name whereas %C means class which would have necessitated location

[mikaylathompson]

Mostly a side note -- it's very weird that Mend decided it didn't need to run on this push (see here). You're adding and updating dependencies!

[AndreKurait]

Does it ever run on a PR, or only once it's merged?

A scan is initiated via a valid GitHub 'push' command. A valid 'push' command...

[mikaylathompson]

Yeah, usually I can see the results when I modify build.gradle files before merging. But it's currently doing this same thing to me on the werkzeug PR.

But definitely not a blocker for this!