Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix default.vw_card_res_char tests to better match the reality of the view #110

Merged
merged 2 commits into from
Aug 31, 2023
Merged

Fix default.vw_card_res_char tests to better match the reality of the view #110

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ab1ef1c
Fix default.vw_card_res_char tests to better match the reality of the…
jeancochrane Aug 31, 2023
8e43ffd
--defer to upstream models in dbt test on CI
jeancochrane Aug 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build_and_test_dbt.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ jobs:
run: |
if [[ $MODIFIED_RESOURCES_ONLY == 'true' ]]; then
echo "Running tests on modified/new resources only"
dbt test -t "$TARGET" -s state:modified state:new --state "$STATE_DIR"
dbt test -t "$TARGET" -s state:modified state:new --defer --state "$STATE_DIR"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fixes a small bug that I hadn't noticed before working on this PR. Prior to this change, the lack of a --defer flag meant that tests would fail when run against views or tables that were not built as part of the CI environment (see here for a sample failing workflow run). The lack of certain views or tables in a CI environment is intentional: We don't want to build models in CI unless their definition is new or has changed on the PR branch, which helps us speed up CI by only building and testing models that are being affected by the PR. The --defer tells dbt that if a model does not exist in the CI version of the DAG, but it does exist in the prod version of the DAG, then it should refer to the prod version for the purpose of running builds and tests. (The exception to this is models that have changed as part of the PR, which are always built in the CI environment due to the -s state:modified flag.)

Detailed docs on dbt's deferral system are here.

else
echo "Running tests on all resources"
dbt test -t "$TARGET"
Expand Down
20 changes: 2 additions & 18 deletions dbt/models/default/schema.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,27 +69,11 @@ models:
tests:
# Unique by card and year
- dbt_utils.unique_combination_of_columns:
name: vw_card_res_char_unique_by_card_and_year
name: vw_card_res_char_unique_by_pin_card_and_year
combination_of_columns:
- pin
- year
- card
config:
error_if: ">913"
wrridgeway marked this conversation as resolved.
Show resolved Hide resolved
# char_recent_renovation correlates with char_renovation
- expression_is_true:
name: vw_card_res_char_renovation_fields_match
# char_renovation can be null, which we count here as falsey
expression: >-
char_recent_renovation = (
case when char_renovation = '1' then true else false end
)
select_columns:
- card
- year
- char_renovation
- char_recent_renovation
config:
error_if: ">73942"
# TODO: Characteristics columns should adhere to pre-determined criteria
- name: default.vw_pin_address
description: '{{ doc("vw_pin_address") }}'
Expand Down