Using log4j2-elasticsearch-hc -> Does AsyncLogger still not support adding dynamic keyValue pairs in Jackson JSONLayout

[gpadbidri]

I want to have my POJO Class as-is in JSON format in the ElasticSearch Index but somehow it is getting stringified to the message property
Is there a way I can overwrite the IndexTemplate to have my format :
This is my XML config :

Following are my Index Template Contents; I dont see the clientID, serviceName etc. fields referred as key-value :

{ "mappings": { "dynamic_templates": [ { "floats": { "match_mapping_type": "long", "mapping": { "type": "float" } } } ], "properties": { "accountId": { "type": "keyword" }, "clientId": { "type": "text" }, "serviceName": { "type": "text" } } } }

[gpadbidri]

OK @rfoltyns I jsut figured out that I can add extra properties to the Elastic Search Index using
<VirtualProperty name="ctxVariable" value="$${ctx:myFavouriteVariable:-notSupportedWithAsyncLogger}" dynamic="true" />

I am using AsyncLogger. How can I set it in code a sthe below does not seem to work :
ThreadContext.put("myFavouriteVariable", UUID.randomUUID().toString());

Does AsyncLogger still not support adding dynamic values to keyValue pairs in Jackson JSONLayout ???

Cos my basic concern is that I want to set my Log POJO as key value in the Elastic Search as opposed to a stringified json set as a value for Message property. Can you tell me how or if I can achieve it ? I am stuck.

[rfoltyns]

This feature is not supported with AsyncLogger unfortunately. It comes down to the way Log4j2 Lookups are hooked into VirtualPropertyWriter.

I'll look into it for 1.7.

In the meantime, if you'd like to log your DTOs as JSON, try:

logger.info(jsonString);

and this

<Elasticsearch messageOnly="true">

This will skip the rest of LogEvent (timeMillis, level, etc.) , so it's a trade-off...

[gpadbidri]

@rfoltyns Thanks for the clarification. What would be the ETA for that feature ? Any ballpark estimates around it ?

[rfoltyns]

Few days to implement and test, but I'll will not be able to focus on it this year.

All accept PRs if you're up for it.

[kocyigityunus]

You can easily achieve this using a painless pipeline processor on elastic.

something similar to:

description: "Convert message to JSON Object",
processors: [
  {
    script: {
      lang: "painless",
      source: `
          try {
            ctx.data = Processors.json(ctx.message);
            ctx.message = ctx.data.message;
          } catch (Exception e) {}
          `,
    },
  },
],

then when logging, just convert it to json string and send

//
val jsonMap = hashMapOf<String, Any>()
jsonMap["message"] = "request.log"
jsonMap["request"] = requestDocument
jsonMap["response"] = responseDocument

//
val jsonString = gson.toJson(jsonMap)
logger.info(jsonString)

Then on your index template mark "parse_json" as "index.final_pipeline" or "index.default_pipeline"