You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We pass an expected output schema into the expression handler. In the expression handler used by the default engines we currently first recuse the current and target schema and look for any mismatches, and then apply the transform, which will cause two full traversals of the data, even if only one small thing is changed.
Instead we could use the initial traversal to mark the highest points in the tree where we know there are no transformations below, and limit the second traversal to the minimal amount of work.
See #331 (comment) for the genesis of this idea and example pseudo-code.
The text was updated successfully, but these errors were encountered:
We pass an expected output schema into the expression handler. In the expression handler used by the default engines we currently first recuse the current and target schema and look for any mismatches, and then apply the transform, which will cause two full traversals of the data, even if only one small thing is changed.
Instead we could use the initial traversal to mark the highest points in the tree where we know there are no transformations below, and limit the second traversal to the minimal amount of work.
See #331 (comment) for the genesis of this idea and example pseudo-code.
The text was updated successfully, but these errors were encountered: