Skip to content

Commit

Permalink
Aggregation layers docs
Browse files Browse the repository at this point in the history
  • Loading branch information
Pessimistress committed Sep 3, 2024
1 parent 2cd0deb commit 89954ee
Show file tree
Hide file tree
Showing 21 changed files with 962 additions and 1,231 deletions.
56 changes: 20 additions & 36 deletions docs/api-reference/aggregation-layers/aggregation-layer.md
Original file line number Diff line number Diff line change
@@ -1,54 +1,38 @@
# AggregationLayer
# AggregationLayer (Experimental)

All of the layers in `@deck.gl/aggregation-layers` module perform some sort of data aggregation. All these layers perform aggregation with different parameters (CPU vs GPU, aggregation to rectangular bins vs hexagon bins, world space vs screen space, aggregation of single weight vs multiple weights etc).
`AggregationLayer` is the base class for all layers in `@deck.gl/aggregation-layers` module. It implements the most common tasks for aggregation with flexibility of customizations.

`AggregationLayer` and `GridAggregationLayer` perform most of the common tasks for aggregation with flexibility of customizations. This document describes what `AggregationLayer` does and how to use it in other aggregation layers.
`AggregationLayer` extends [CompositeLayer](../core/composite-layer.md).

## Methods

`AggregationLayer` is subclassed form `CompositeLayer` and all layers in `@deck.gl/aggregation-layers` are subclassed from this Layer.
Any layer subclassing the `AggregationLayer` must implement the following methods:

## Integration with `AttributeManager`
#### `getAggregatorType` {#getaggregatortype}

This layer creates `AttributeManager` and makes it available for its subclasses. Any aggregation layer can add attributes to the `AttributeManager` and retrieve them using `getAttributes` method. This enables using `AttributeManager`'s features and optimization for using attributes. Also manual iteration of `data` prop can be removed and attributes can be directly set on GPU aggregation models or accessed directly for CPU aggregation.
Returns a string that indicates the type of aggregator that this layer uses, for example `'gpu'`. This method is invoked with each `updateState` lifecycle. If the type string does not match its previous value, the existing aggregator will be disposed and recreated.

Example: Adding attributes to an aggregation layer
#### `createAggregator` {#createaggregator}

```
const attributeManager = this.getAttributeManager();
attributeManager.add({
positions: {size: 3, accessor: 'getPosition'},
color: {size: 3, accessor: 'getColorWeight'},
elevation: {size: 3, accessor: 'getElevationWeight'}
});
```
Arguments:
- `type` (string) - return value from `getAggregatorType()`

## updateState()
Returns a [Aggregator](./aggregator.md) instance. This instance will be accessible via `this.state.aggregator`.

During update state, Subclasses of `AggregationLayer` must first call 'super.updateState()', which calls
#### `onAttributeChange` {#onattributechange}

- `updateShaders(shaders)` : Subclasses can override this if they need to update shaders, for example, when performing GPU aggregation, aggregation shaders must be merged with argument of this function to correctly apply `extensions`.
Arguments:
- `attributeId` (string) - the id of an attribute that has been updated

- `_updateAttributes`: This checks and updates attributes based on updated props.
This event handler should be used to update the props of the aggregator, if needed, and call `aggregator.setNeedsUpdate` to request an update.

## Checking if aggregation is dirty
#### `renderLayers` {#renderlayers}

### Dimensions
Returns a list of sub layers.

Typical aggregation, involves :
1. Group the input data points into bins
2. Compute the aggregated value for each bin
Aggregation results can be obtained here with `aggregator.getBins`, `aggregator.getResult` and `aggregator.getResultDomain`.

For example, when `cellSize` or `data` is changed, layer needs to perform both `1` and `2` steps, when a parameter affecting a bin's value is changed (like `getWeight` accessor), layer only need to perform step `2`.

When doing CPU Aggregation, both above steps are performed individually. But for GPU aggregation, both are merged into single render call.
## Source

To support what state is dirty, constructor takes `dimensions` object, which contains, several keyed dimensions. It must contain `data` dimension that defines, when re-aggregation needs to be performed.

### isAggregationDirty()

This helper can be used if a dimension is changed. Sublayers can defined custom dimensions and call this method to check if a dimension is changed.


### isAttributeChanged()

`AggregationLayer` tracks what attributes are changed in each update cycle. Super classes can use `isAttributeChanged()` method to check if a specific attribute is changed or any attribute is changed.
[modules/aggregation-layers/src/common/aggregation-layer.ts](https://github.com/visgl/deck.gl/tree/master/modules/aggregation-layers/src/common/aggregation-layer.ts)
168 changes: 168 additions & 0 deletions docs/api-reference/aggregation-layers/aggregator.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
# Aggregator Interface

The `Aggregator` interface describes a type of class that performs aggregation.

## Terminology

_Aggregation_ is a 2-step process:

1. **Sort**: Group a collection of _data points_ by some property into bins_.
2. **Aggregate**: for each _bin_, calculate a numeric output (_result_) from some metrics (_values_) from all its members. Multiple results can be obtained independently (_channels_).

An implementation of the _Aggregator_ interface takes the following inputs:
- The number of data points
- The group that each data point belongs to, by mapping each data point to a _binId_ (array of integers)
- The values to aggregate, by mapping each data point in each channel to one _value_ (number)
- The method (_operation_) to reduce a list of values to one number, such as SUM

And yields the following outputs:
- A list of _binIds_ that data points get sorted into
- The aggregated values (_result_) as a list of numbers, comprised of one number per bin per channel
- The [min, max] among all aggregated values (_domain_) for each channel

### Example

Consider the task of making a [histogram](https://en.wikipedia.org/wiki/Histogram) that shows the result of a survey by age distribution.

1. The _data points_ are the list of participants, and we know the age of each person.
2. Suppose we want to group them by 5-year intervals. A 21-year-old participant is assigned to the bin of age 20-25, with _binId_ `[20]`. A 35-year-old participant is assigned to the bin of age 35-40, with _binId_ `[35]`, and so on.
3. For each bin (i.e. age group), we calculate 2 _values_:
+ The first _channel_ is "number of participants". Each participant in this group yields a _value_ of 1, and the result equals all values added together (_operation_: SUM).
+ The second _channel_ is "average score". Each participant in this group yields a _value_ that is their test score, and the result equals the sum of all scores divided by the number of participants (_operation_: MEAN).
4. As the outcome of the aggregation, we have:
+ Bins: `[15, 20, 25, 30, 35, 40]`
+ Channel 0 result: `[1, 5, 12, 10, 8, 3]`
+ Channel 0 domain: `[1, 12]`
+ Channel 1 result: `[6, 8.2, 8.5, 7.9, 7.75, 8]`
+ Channel 1 domain: `[6, 8.5]`


## Methods

An implementation of `Aggregator` should expose the following methods:

#### `setProps` {#setprops}

Set runtime properties of the aggregation.

```ts
aggregator.setProps({
pointCount: 10000,
attributes: {...},
operations: ['SUM', 'MEAN'],
binOptions: {groupSize: 5}
});
```

Arguments:
- `pointCount` (number) - number of data points.
- `attributes` ([attributes](../core/layer.md#dataattributes)) - the input data in binary format.
- `operations` (string[]) - How to aggregate the values inside a bin, defined per channel.
- `binOptions` (object) - arbitrary settings that affect bin sorting.
- `onUpdate` (Function) - callback when a channel has been recalculated. Receives the following arguments:
+ `channel` (number) - the channel that just updated

#### `setNeedsUpdate` {#setneedsupdate}

Flags a channel to need update. This could be a result of change in the input data or bin options.

```ts
aggregator.setNeedsUpdate(0);
```

Arguments:
- `channel` (number, optional) - mark the given channel as dirty. If not provided, all channels will be updated.

#### `update` {#update}

Called after all props are set and before results are accessed. The aggregator should allocate resources and redo aggregations if needed at this stage.

```ts
aggregator.update();
```

#### `preDraw` {#predraw}

Called before the result buffers are drawn to screen. Certain types of aggregations are dependent on render time context and this is alternative opportunity to update just-in-time.

```ts
aggregator.preDraw({moduleSettings: ...});
```

#### `getBin` {#getbin}

Get the information of a given bin.

```ts
const bin = aggregator.getBin(100);
```

Arguments:
- `index` (number) - index of the bin to locate it in `getBins()`

Returns:
- `id` (number[]) - Unique bin ID.
- `value` (number[]) - Aggregated values by channel.
- `count` (number) - Number of data points in this bin.
- `pointIndices` (number[] | undefined) - Indices of data points in this bin if available. This field may not be populated when using GPU-based implementations.

#### `getBins` {#getbins}

Get an accessor to all bin IDs.

```ts
const binIdsAttribute = aggregator.getBins();
```

Returns:
- A [binary attribute](../core/layer.md#dataattributes) of the output bin IDs, or
- null, if `update` has never been called

#### `getResult` {#getresult}

Get an accessor to the aggregated values of a given channel.

```ts
const resultAttribute = aggregator.getResult(0);
```

Arguments:
- `channel` (number) - the channel to retrieve results from

Returns:
- A [binary attribute](../core/layer.md#dataattributes) of the output values of the given channel, or
- null, if `update` has never been called

#### `getResultDomain` {#getresultdomain}

Get the [min, max] of aggregated values of a given channel.

```ts
const [min, max] = aggregator.getResultDomain(0);
```

Arguments:
- `channel` (number) - the channel to retrieve results from

Returns the domain ([number, number]) of the aggregated values of the given channel.

#### `destroy` {#destroy}

Dispose all allocated resources.

```ts
aggregator.destroy();
```


## Members

An implementation of `Aggregator` should expose the following members:

#### `binCount` (number) {#bincount}

The number of bins in the aggregated result.

## Source

[modules/aggregation-layers/src/common/aggregator/aggregator.ts](https://github.com/visgl/deck.gl/tree/master/modules/aggregation-layers/src/common/aggregator/aggregator.ts)
35 changes: 25 additions & 10 deletions docs/api-reference/aggregation-layers/contour-layer.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ npm install @deck.gl/core @deck.gl/layers @deck.gl/aggregation-layers

```ts
import {ContourLayer} from '@deck.gl/aggregation-layers';
import type {ContourLayerProps} from '@deck.gl/aggregation-layers';
import type {ContourLayerProps, ContourLayerPickingInfo} from '@deck.gl/aggregation-layers';

new ContourLayer<DataT>(...props: ContourLayerProps<DataT>[]);
```
Expand All @@ -171,7 +171,7 @@ new deck.ContourLayer({});

Inherits from all [Base Layer](../core/layer.md) properties.

### Render Options
### Aggregation Options

#### `cellSize` (number, optional) ![transition-enabled](https://img.shields.io/badge/transition-enabled-green.svg?style=flat-square") {#cellsize}

Expand All @@ -183,19 +183,27 @@ Size of each cell in meters

* Default: true

When set to true and browser supports GPU aggregation, aggregation is performed on GPU. GPU aggregation can be 2 to 3 times faster depending upon number of points and number of cells.
When set to `true` and the browser supports it, aggregation is performed on GPU.

In the right context, enabling GPU aggregation can significantly speed up your application. However, depending on the nature of input data and required application features, there are pros and cons in leveraging this functionality. See [CPU vs GPU aggregation](./overview.md#cpu-vs-gpu-aggregation) for an in-depth discussion.


#### `aggregation` (string, optional) {#aggregation}

* Default: 'SUM'
* Default: `'SUM'`

Defines the operation used to aggregate all data object weights to calculate a cell's value. Valid values are:

- `'SUM'`: The sum of weights across all points that fall into a cell.
- `'MEAN'`: The mean weight across all points that fall into a cell.
- `'MIN'`: The minimum weight across all points that fall into a cell.
- `'MAX'`: The maximum weight across all points that fall into a cell.
- `'COUNT'`: The number of points that fall into a cell.

Defines the type of aggregation operation, valid values are 'SUM', 'MEAN', 'MIN' and 'MAX'. When no value or an invalid value is set, 'SUM' is used as aggregation.
`getWeight` and `aggregation` together determine the elevation value of each cell.

* SUM : Grid cell contains sum of all weights that fall into it.
* MEAN : Grid cell contains mean of all weights that fall into it.
* MIN : Grid cell contains minimum of all weights that fall into it.
* MAX : Grid cell contains maximum of all weights that fall into it.

### Render Options

#### `contours` (object[], optional) {#contours}

Expand Down Expand Up @@ -240,11 +248,18 @@ The weight of each object.
* If a function is provided, it is called on each object to retrieve its weight.


## Picking

The [PickingInfo.object](../../developer-guide/interactivity.md#the-pickinginfo-object) field returned by hover/click events of this layer represents a path (isoline) or a polygon (isoband). The object contains the following fields:

- `contour` (object) - one of the contour configurations passed to the `contours` prop.


## Sub Layers

The `ContourLayer` renders the following sublayers:

* `lines` - For Isolines, rendered by [LineLayer](../layers/line-layer.md)
* `lines` - For Isolines, rendered by [PathLayer](../layers/path-layer.md)
* `bands` - For Isobands, rendered by [SolidPolygonLayer](../layers/solid-polygon-layer.md)


Expand Down
Loading

0 comments on commit 89954ee

Please sign in to comment.