Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Auditor improvements and optimizations #802

Open
EddieLF opened this issue May 27, 2024 · 0 comments
Open

Auditor improvements and optimizations #802

EddieLF opened this issue May 27, 2024 · 0 comments
Assignees
@EddieLF
Labels
bug Something isn't working enhancement New feature or request

Comments

@EddieLF
Copy link
Contributor

EddieLF commented May 27, 2024 •
edited
Loading

The auditing module was created to scan the upload bucket & query metamist to work out which files can be safely deleted and which files should be ingested.

It does its job fairly well, however some queries are unoptimized and failing for large datasets. E.g.
https://batch.hail.populationgenomics.org.au/batches/454428/jobs/1
https://batch.hail.populationgenomics.org.au/batches/454440/jobs/1
https://batch.hail.populationgenomics.org.au/batches/454445/jobs/1
https://batch.hail.populationgenomics.org.au/batches/454450/jobs/1
@EddieLF EddieLF changed the title Auditor improvments and optimizations Auditor improvements and optimizations May 27, 2024
@EddieLF EddieLF self-assigned this May 27, 2024
@EddieLF EddieLF added bug Something isn't working enhancement New feature or request labels May 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@EddieLF EddieLF

Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@EddieLF