Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Time Travel Read test #8

Open
wjones127 opened this issue Oct 4, 2022 · 2 comments
Open

Time Travel Read test #8

wjones127 opened this issue Oct 4, 2022 · 2 comments

Comments

@wjones127
Copy link
Collaborator

Correctly resolving time travel is an important part of Delta Lake. One way to test that is to add additional parquet files to the expected data. Ones that aren't the current version could have files names such as expected_data_{i}.parquet where {i} is the Delta table version number.

@scottsand-db
Copy link

scottsand-db commented Oct 18, 2022

└── reference_tables
    ├── reference_table_1
    │   |── delta
    │   │   └── a.parquet
    │   │   └── b.parquet
    │   │   └── c.parquet
    │   │   └── _delta_log
    │   ├── latest
    │   │   └── a.parquet
    │   │   └── b.parquet
    │   │   └── c.parquet
    │   ├── v0
    │   │   └── a.parquet
    │   └── v1
    │   │   └── a.parquet
    │   │   └── b.parquet
    └── reference_table_2

@wjones127
Copy link
Collaborator Author

wjones127 commented Oct 18, 2022

latest expected table is required, versions ones (v1) are optional.

/reference_tables
  /reference_table_1
    /delta
      /_delta_log
      parquet
    /expected
      /latest
        parquet
      /v1
        parquet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants