Add Dockerfile and `build-and-run-model` workflow for CI model runs #9

jeancochrane · 2023-11-09T21:35:58Z

This PR adds a new build-and-run-model workflow to build, run, and cleanup the model on GitHub Actions. The workflow is configured to run on PRs and workflow dispatch. On pushes to the main branch, the workflow will rebuild the container image for the main branch, but will not run or cleanup the model.

This PR mirrors the following PRs in model-res-avm:

Add Dockerfile and workflow for building and pushing Docker image to GHCR model-res-avm#25
Refactor model workflow to use external workflow defined in ccao-data/actions model-res-avm#59

Note also that in order to test a full model run, we switch to using the loc_cook_municipality_name feature instead of loc_tax_municipality_name. This is not actually the correct feature, and we'll need to switch it back in #4, but loc_tax_municipality_name is not yet present in the training data and properly refreshing the training data is outside the scope of this issue.

Closes #5.

Testing

This PR is pinned to the main branch of the ccao-data/actions repo, which will not contain the reusable workflow code that this branch needs until ccao-data/actions#1 gets merged. This means that the most recent build-and-run-model run will appear not to be successful. For an example of a successful model run, see this workflow which was run on the corresponding feature branch of ccao-data/actions containing the reusable workflow code that this repo needs.

…nd-run-batch-job

This reverts commit 299b40b.

This reverts commit a5a64f5.

jeancochrane · 2023-11-13T22:21:12Z

.github/workflows/build-and-run-model.yaml

+    with:
+      vcpu: "16.0"
+      memory: "65536"
+      role-duration-seconds: 14400  # Worst-case time for a full model run


This is probably not correct, but I'm setting it to match the value for the res model for now until we improve model performance and get a better sense of our worst-case times.

README.Rmd

jeancochrane · 2023-11-13T22:24:18Z

dvc.yaml

@@ -47,7 +47,7 @@ stages:
        cache: false
    - output/workflow/recipe/model_workflow_recipe.rds:
        cache: false
-    frozen: true


I'm pretty sure this step was erroneously marked as frozen; we don't cache intermediate outputs so freezing the train step leads to an error. Billy seemed to think this was reasonable but let me know if I'm off base!

This is definitely an error. The only step that should really ever be frozen is the ingest step.

jeancochrane · 2023-11-13T22:25:29Z

pipeline/05-finalize.R

@@ -9,6 +9,7 @@
 suppressPackageStartupMessages({
  library(arrow)
  library(aws.s3)
+  library(aws.ec2metadata)


As we discovered in ccao-data/model-res-avm#26, this package is necessary in order to allow the aws.s3 package to authenticate using credentials in an ECS environment.

This reverts commit 82c3081.

This reverts commit 33f83a5.

…y_name" This reverts commit 82d6155.

jeancochrane · 2023-11-14T17:57:11Z

README.md

@@ -152,12 +152,12 @@ the 2023 assessment model.
 | Percent Population Mobility, Moved From Within Same County in Past Year | acs5           | numeric   |                       |
 | Longitude                                                               | loc            | numeric   |                       |
 | Latitude                                                                | loc            | numeric   |                       |
+| Municipality Name                                                       | loc            | character |                       |


This is just an artifact of removing the var_name_model != "loc_tax_municipality_name" conditional in README.Rmd above. I'm not sure why it changes the order of the feature in the table, but it represents the reverse of what happened in #4.

jeancochrane · 2023-11-14T17:57:44Z

README.Rmd

@@ -106,11 +106,9 @@ ccao::vars_dict %>%
    )
  ) %>%
  mutate(`Unique to Condo Model` = ifelse(
-    var_name_model != "loc_tax_municipality_name" & (


We no longer need this conditional now that both repos are (temporarily) using loc_cook_municipality_name.

…ctions"" This reverts commit e801405.

jeancochrane · 2023-11-14T18:31:00Z

Oops, I forgot to pin to the main branch of the build-and-run-batch-job action before requesting review! Done in a852f22, so this should be ready for review.

dfsnow

Looks good @jeancochrane. Let's get merges going for the actions repo + this, then do test runs of the res + condo pipelines just to make sure everything is working.

dfsnow · 2023-11-14T19:00:41Z

dvc.yaml

@@ -47,7 +47,7 @@ stages:
        cache: false
    - output/workflow/recipe/model_workflow_recipe.rds:
        cache: false
-    frozen: true


This is definitely an error. The only step that should really ever be frozen is the ingest step.

…o-data/actions""" This reverts commit a852f22.

…h of ccao-data/actions"""" This reverts commit 3f2114e.

jeancochrane added 3 commits November 9, 2023 15:31

Simplify Pipenv dependencies

06a7bc9

Add Dockerfile

8eb4fa7

Add build-and-run-model workflow

36e84e5

jeancochrane linked an issue Nov 9, 2023 that may be closed by this pull request

[Infra updates] Copy res model infra updates to the condo model #5

Closed

jeancochrane had a problem deploying to deploy November 9, 2023 21:42 — with GitHub Actions Failure

jeancochrane added 2 commits November 9, 2023 16:12

Temporarily pin build-and-run-model to development version of build-a…

1d8bb4a

…nd-run-batch-job

Temporarily restrict dvc repro to the train step, for testing

299b40b

jeancochrane force-pushed the jeancochrane/5-infra-updates-copy-res-model-infra-updates-to-the-condo-model branch from 82518a4 to 299b40b Compare November 9, 2023 22:12

jeancochrane had a problem deploying to deploy November 9, 2023 22:13 — with GitHub Actions Failure

Revert "Temporarily restrict dvc repro to the train step, for testing"

f2391db

This reverts commit 299b40b.

jeancochrane had a problem deploying to deploy November 9, 2023 22:31 — with GitHub Actions Failure

Unfreeze train step in dvc.yaml

d9b0712

jeancochrane had a problem deploying to deploy November 9, 2023 22:39 — with GitHub Actions Failure

jeancochrane had a problem deploying to deploy November 9, 2023 22:47 — with GitHub Actions Failure

Fix typo in loc_cook_municipality_name feature in params.yaml

7fff710

jeancochrane force-pushed the jeancochrane/5-infra-updates-copy-res-model-infra-updates-to-the-condo-model branch from 3a182bd to 7fff710 Compare November 9, 2023 22:55

jeancochrane had a problem deploying to deploy November 9, 2023 22:56 — with GitHub Actions Failure

Update README to reflect change in municipality name feature

2315f8e

jeancochrane had a problem deploying to deploy November 9, 2023 23:06 — with GitHub Actions Failure

Lint README.Rmd

8bd4983

jeancochrane had a problem deploying to deploy November 13, 2023 20:00 — with GitHub Actions Failure

Add print statements to finalize step to debug S3 failures

a5a64f5

jeancochrane force-pushed the jeancochrane/5-infra-updates-copy-res-model-infra-updates-to-the-condo-model branch from 8e5a417 to a5a64f5 Compare November 13, 2023 20:40

jeancochrane had a problem deploying to deploy November 13, 2023 20:41 — with GitHub Actions Failure

jeancochrane added 2 commits November 13, 2023 21:39

Add aws.ec2metadata packaage so that ECS can authenticate with S3

190210f

Revert "Add print statements to finalize step to debug S3 failures"

e11dfb1

This reverts commit a5a64f5.

jeancochrane temporarily deployed to deploy November 13, 2023 21:44 — with GitHub Actions Inactive

Pin build-and-run-model to main branch of ccao-data/actions

82c3081

jeancochrane commented Nov 13, 2023

View reviewed changes

Rename loc_cook_municipality_name back to loc_tax_municipality_name

82d6155

Revert "Pin build-and-run-model to main branch of ccao-data/actions"

e801405

This reverts commit 82c3081.

jeancochrane had a problem deploying to deploy November 13, 2023 22:52 — with GitHub Actions Failure

jeancochrane and others added 3 commits November 14, 2023 16:16

Add ingest dependencies to renv.lock

33f83a5

Revert "Add ingest dependencies to renv.lock"

4756f53

This reverts commit 33f83a5.

Revert "Rename loc_cook_municipality_name back to loc_tax_municipalit…

41ebdfb

…y_name" This reverts commit 82d6155.

jeancochrane temporarily deployed to deploy November 14, 2023 17:37 — with GitHub Actions Inactive

jeancochrane commented Nov 14, 2023

View reviewed changes

jeancochrane marked this pull request as ready for review November 14, 2023 18:29

jeancochrane requested review from dfsnow and wrridgeway as code owners November 14, 2023 18:29

Revert "Revert "Pin build-and-run-model to main branch of ccao-data/a…

a852f22

…ctions"" This reverts commit e801405.

dfsnow approved these changes Nov 14, 2023

View reviewed changes

Revert "Revert "Revert "Pin build-and-run-model to main branch of cca…

3f2114e

…o-data/actions""" This reverts commit a852f22.

jeancochrane had a problem deploying to deploy November 14, 2023 21:09 — with GitHub Actions Failure

Revert "Revert "Revert "Revert "Pin build-and-run-model to main branc…

fd2fcdd

…h of ccao-data/actions"""" This reverts commit 3f2114e.

jeancochrane merged commit 6cdc38f into master Nov 14, 2023
1 check passed

jeancochrane deleted the jeancochrane/5-infra-updates-copy-res-model-infra-updates-to-the-condo-model branch November 14, 2023 22:44

jeancochrane mentioned this pull request Nov 14, 2023

Bugfix: Pin build-and-run-batch-job workflow ref to main #11

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Dockerfile and `build-and-run-model` workflow for CI model runs #9

Add Dockerfile and `build-and-run-model` workflow for CI model runs #9

jeancochrane commented Nov 9, 2023 •

edited

Loading

jeancochrane Nov 13, 2023

jeancochrane Nov 13, 2023

dfsnow Nov 14, 2023

jeancochrane Nov 13, 2023

jeancochrane Nov 14, 2023

jeancochrane Nov 14, 2023

jeancochrane commented Nov 14, 2023

dfsnow left a comment

dfsnow Nov 14, 2023

Add Dockerfile and build-and-run-model workflow for CI model runs #9

Add Dockerfile and build-and-run-model workflow for CI model runs #9

Conversation

jeancochrane commented Nov 9, 2023 • edited Loading

Testing

jeancochrane Nov 13, 2023

Choose a reason for hiding this comment

jeancochrane Nov 13, 2023

Choose a reason for hiding this comment

dfsnow Nov 14, 2023

Choose a reason for hiding this comment

jeancochrane Nov 13, 2023

Choose a reason for hiding this comment

jeancochrane Nov 14, 2023

Choose a reason for hiding this comment

jeancochrane Nov 14, 2023

Choose a reason for hiding this comment

jeancochrane commented Nov 14, 2023

dfsnow left a comment

Choose a reason for hiding this comment

dfsnow Nov 14, 2023

Choose a reason for hiding this comment

Add Dockerfile and `build-and-run-model` workflow for CI model runs #9

Add Dockerfile and `build-and-run-model` workflow for CI model runs #9

jeancochrane commented Nov 9, 2023 •

edited

Loading