-
Notifications
You must be signed in to change notification settings - Fork 4
Data lifecycle and performance considerations
The KustoLoco framework expects data and associated structures to be immutable with a few notable exceptions. If this expectation is violated "bad things may happen"(TM)...
Specifically:
There is an assumption that cells within columns are immutable. If you create a table from a set of records, you must ensure you do not subsequently mutate those records. The default mechanism for creating record-based tables is to use a LambdaWrappedColumn that uses the property getters. Therefore properties are expected to be both immutable and cheap to access. If this is not the case, user code should either copy data into an immutable set of records or use the older alloc-based ColumnBuilder mechanism
Columns are effectively immutable and this is enforced by the engine. Columns may be cheaply shared or sliced across tables
Table are effectively immutable and this is enforced by the engine. It is possible to cheaply create copies/slices of tables
The KustoQueryContext is NOT immutable. (The clue is in the name!) The context holds the current set of tables and other state against which multiple queries may be issued. The context allows tables to be added, removed, copied etc but this may only occur between queries. KustoQueryContext does not enforce any thread-safety; it's assumed that client code will always pipeline table manipulation and query issuance correctly.
In the particular case of lazy-loaded tables, it is up to the loader to ensure it does not attempt to add tables simultaneously from multiple threads
The query result is immutable and is effectively a table along with other information. (In fact it's possible to materialize a QueryResult back into a context as a named table.)