[DOC] Add ingest processors

[vagimeli]

What do you want to do?

• Request a change to existing documentation
• Add new documentation
• Report a technical problem with the documentation
• Other

Tell us about your request. Provide a summary of the request and all versions that are affected.
Add ingest processors; new content, several processors

What other resources are available? Provide links to related issues, POCs, steps for testing, etc.

[vagimeli]

Move ingest pipelines docs to new section - Ingest Pipelines - under Managing Indexes; combine the four pipeline docs into one (index page); add processors section under Ingest Pipelines; create new pages for all OpenSearch processors.

curl -X GET "localhost:9200/_nodes/ingest?pretty"
{
"_nodes" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"cluster_name" : "opensearch-cluster",
"nodes" : {
"i9hLyCxYTqOpa22BcJkKyg" : {
"name" : "opensearch-node1",
"transport_address" : "172.18.0.3:9300",
"host" : "172.18.0.3",
"ip" : "172.18.0.3",
"version" : "2.7.0",
"build_type" : "tar",
"build_hash" : "b7a6e09e492b1e965d827525f7863b366ef0e304",
"roles" : [
"cluster_manager",
"data",
"ingest",
"remote_cluster_client"
],
"attributes" : {
"shard_indexing_pressure_enabled" : "true"
},
"ingest" : {
"processors" : [
{
"type" : "append"
},
{
"type" : "bytes"
},
{
"type" : "convert"
},
{
"type" : "csv"
},
{
"type" : "date"
},
{
"type" : "date_index_name"
},
{
"type" : "dissect"
},
{
"type" : "dot_expander"
},
{
"type" : "drop"
},
{
"type" : "fail"
},
{
"type" : "foreach"
},
{
"type" : "geoip"
},
{
"type" : "geojson-feature"
},
{
"type" : "grok"
},
{
"type" : "gsub"
},
{
"type" : "html_strip"
},
{
"type" : "join"
},
{
"type" : "json"
},
{
"type" : "kv"
},
{
"type" : "lowercase"
},
{
"type" : "pipeline"
},
{
"type" : "remove"
},
{
"type" : "rename"
},
{
"type" : "script"
},
{
"type" : "set"
},
{
"type" : "sort"
},
{
"type" : "split"
},
{
"type" : "text_embedding"
},
{
"type" : "trim"
},
{
"type" : "uppercase"
},
{
"type" : "urldecode"
},
{
"type" : "user_agent"
}
]
}
}
}
}

[hdhalter]

Hi Melissa, I just linked a related issue that was reported by a community member regarding missing processors.

[hdhalter]

• append-processor
• bytes-processor
• ingest-circle-processor
• convert-processor
• date-index-name-processor
• dissect-processor
• dot-expand-processor
• drop-processor
• fail-processor
• foreach-processor
• geoip-processor
• gsub-processor
• htmlstrip-processor
• inference-processor
• join-processor
• kv-processor
• pipeline-processor
• remove-processor
• rename-processor
• script-processor
• set-processor
• ingest-node-set-security-user-processor
• sort-processor
• urldecode-processor
• user-agent-processor

[hdhalter]

@vagimeli - do we need to keep this issue open until all the ingest processors are documented?

[hdhalter]

Ingest processors are tracked here: #4647