Skip to content

Commit

Permalink
Fix style-job comments
Browse files Browse the repository at this point in the history
Signed-off-by: Rishabh Maurya <rishabhmaurya05@gmail.com>
  • Loading branch information
rishabhmaurya committed Jun 7, 2024
1 parent d55464d commit d6ddd27
Showing 1 changed file with 19 additions and 43 deletions.
62 changes: 19 additions & 43 deletions _field-types/supported-field-types/derived-field.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ redirect_from:
- /field-types/derived/
---

# Derived field type
## Derived field type

Derived fields enable users to create new fields dynamically by executing scripts on existing fields retrieved from the `_source` field, which contains the original document, or from a field's doc values for faster retrieval. Once defined, either in the index mapping or within a search request, these fields can be utilized like regular fields in query definitions.

Expand All @@ -37,8 +37,6 @@ Currently, derived fields have the following limitations:

These limitations are recognized, and there are plans to address them in future releases.

# Getting started

## Prerequisites
- **Enable `_source` or Doc Values**: Ensure that either the `_source` field or doc values are enabled for the fields used in your script.
- **Enable expensive queries**: Ensure [`search.allow_expensive_queries`]({{site.url}}{{site.baseurl}}/query-dsl/index/#expensive-queries) is set to `true`.
Expand Down Expand Up @@ -73,28 +71,6 @@ PUT logs
```
{% include copy-curl.html %}

```json
PUT logs
{
"mappings": {
"properties": {
"request": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"client_ip": {
"type": "keyword"
}
}
}
}
```
{% include copy-curl.html %}

```json
POST _bulk
{ "index" : { "_index" : "logs", "_id" : "1" } }
Expand Down Expand Up @@ -175,24 +151,24 @@ PUT /logs/_mapping

All parameters are dynamic and can be modified without the need to reindex.

#### Emitting values in script
### Emitting values in script

The `emit()` function is available only within the derived field script context. It is used to emit one or more values (for a multi-valued field) from the script for a given document on which the script runs.

Here is the emit format and support for multi-valued fields for different types:

| Type | Emit Format | Multi-valued |
|-----------|-----------------------------------|--------------|
| `date` | `emit(long timeInMilis)` | Yes |
| `geo_point`| `emit(double lat, double lon)` | Yes |
| `ip` | `emit(String ip)` | Yes |
| `keyword` | `emit(String)` | Yes |
| `text` | `emit(String)` | Yes |
| `long` | `emit(long)` | Yes |
| `double` | `emit(double)` | Yes |
| `float` | `emit(float)` | Yes |
| `boolean` | `emit(boolean)` | No |
| `object` | `emit(String json)` (valid JSON) | Yes |
| Type | Emit format | Multi-valued |
|-----------|----------------------------------|--------------|
| `date` | `emit(long timeInMilis)` | Yes |
| `geo_point`| `emit(double lat, double lon)` | Yes |
| `ip` | `emit(String ip)` | Yes |
| `keyword` | `emit(String)` | Yes |
| `text` | `emit(String)` | Yes |
| `long` | `emit(long)` | Yes |
| `double` | `emit(double)` | Yes |
| `float` | `emit(float)` | Yes |
| `boolean` | `emit(boolean)` | No |
| `object` | `emit(String json)` (valid JSON) | Yes |

In case of a mismatch between the `type` of the derived field and its emitted value, it will result in an `IllegalArgumentException`, and the search request will fail. However, if `ignore_malformed` is set, the document for which the failure occurred will be skipped, and the search request will not fail.

Expand Down Expand Up @@ -236,7 +212,7 @@ POST /logs/_search
```
{% include copy-curl.html %}

### Defining mappings in search request
## Defining mappings in search request
You can also define derived fields directly in a search request and query on them in conjunction with regular indexed fields. Here's an example:

```json
Expand Down Expand Up @@ -293,7 +269,7 @@ POST /logs/_search
Fields can be retrieved using the `fields` parameter in the search request, similar to regular fields as shown in the preceding examples. Wildcards can also be used to retrieve all derived fields that match a given pattern.


### Highlight
**Highlight**
For derived fields of type `text` where highlighting makes sense, the currently supported highlighter is the [Unified Highlighter]({{site.url}}{{site.baseurl}}/opensearch/search/highlight#the-unified-highlighter)

```json
Expand Down Expand Up @@ -335,14 +311,14 @@ POST /logs/_search
```
{% include copy-curl.html %}

# Performance
Derived fields are not indexed and are computed on-the-fly by retrieving values from _source field or doc values. Consequently, they can be slow to execute. To improve performance:
## Performance
Derived fields are not indexed and are computed on-the-fly by retrieving values from `_source` field or doc values. Consequently, they can be slow to execute. To improve performance:

- Prune the search space by adding query filters on indexed fields in conjunction with derived fields.
- Use doc values in the script wherever available for faster access compared to `_source`.
- Consider using `prefilter_field` to automatically prune the search space without explicit filters in the search request.

### Prefilter field`
### Prefilter field
This technique helps prune the search space automatically without adding explicit filters in the search request. It implicitly adds a filter on the specified indexed field (`prefilter_field`) when constructing the query. `prefilter_field ` must be of text family types (`text`, `match_only_text`).

For example, lets update the mapping for derived field `method` with `"prefilter_field": "request"`:
Expand Down

0 comments on commit d6ddd27

Please sign in to comment.