Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: one json column to rule them all #23

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

gitcommitshow
Copy link

@gitcommitshow gitcommitshow commented Mar 22, 2024

RudderStack discards events with more than 200 properties as "Bad Event". This results in droping genuine events sometimes. Also these properties are flattened by default in warehouse, resulting in too many columns sometimes reaching the upper limit of columns in warehouses. A usual challenge faced in collecting data from unstructured or semi-structured webhook data with lots of nested values.

This user transformation is a solution to these challenges.
It enables your data pipeline to store all event properties as single json data type column.
Here, we first move event.properties to context.properties and then set jsonPaths option to track.context.proprerties. This instructs RudderStack to not flatten context.proprerties of track events and store that as json data type column.

More info - https://www.rudderstack.com/docs/destinations/warehouse-destinations/json-column-support/

A solution to comply with event property limit of 200
json docs link and change filename
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant