Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Infra updates] Create a manually dispatched GitHub Actions workflow to delete model runs #56

Closed
Tracked by #175
dfsnow opened this issue Nov 3, 2023 · 1 comment
Assignees

Comments

@dfsnow
Copy link
Member

dfsnow commented Nov 3, 2023

Not every run created by the modeling pipeline(s) actually needs to be kept. Many runs are testing some sort of CI or pipeline infrastructure change and aren't serious candidates for model selection. These runs pollute the Athena model.* tables and cost S3 storage. We should delete them when possible.

Manually deleting all the artifacts of a model run from the relevant S3 buckets is kind of a pain, so in the past we used a helper function (https://github.com/ccao-data/model-res-avm/blob/master/R/helpers.R#L35-L64) to delete unneeded runs.

This year, to make things easier, we should create a dedicated GitHub Actions workflow to delete erroneous model runs. The workflow should meet the following requirements:

  • Manual dispatch only with a deployment environment check
  • Takes a model run_id as an input
  • Tightly scoped IAM permissions to only allow recent (>= 2024) runs to be deleted
@dfsnow dfsnow added this to the Updated model pipeline MVP milestone Nov 3, 2023
@dfsnow dfsnow changed the title [Infra updates] Create a manually dispatch GitHub Actions workflow to delete model runs [Infra updates] Create a manually dispatched GitHub Actions workflow to delete model runs Nov 14, 2023
@jeancochrane jeancochrane self-assigned this Nov 16, 2023
@jeancochrane
Copy link
Contributor

Closed by #60.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants