Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sites table generation #692

Draft
wants to merge 32 commits into
base: main
Choose a base branch
from
Draft

Sites table generation #692

wants to merge 32 commits into from

Commits on Apr 16, 2024

  1. changing linting

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    7e4c53c View commit details
    Browse the repository at this point in the history

  2. linting fixes

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    ad7be53 View commit details
    Browse the repository at this point in the history

  3. testing initial sites generation on 100 exomes

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    dba0e17 View commit details
    Browse the repository at this point in the history

  4. import fix

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    774e1e9 View commit details
    Browse the repository at this point in the history

  5. passing output path as string

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    1d08552 View commit details
    Browse the repository at this point in the history

  6. debugging statements

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    1e374d9 View commit details
    Browse the repository at this point in the history

  7. more print statements

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    051bf2d View commit details
    Browse the repository at this point in the history

  8. changing from output_path to just string

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    1b27fe8 View commit details
    Browse the repository at this point in the history

  9. corrected filtering

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    65ed48f View commit details
    Browse the repository at this point in the history

  10. adding init_batch()

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    bcad42b View commit details
    Browse the repository at this point in the history

  11. adding init_batch() to main

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    c113e4a View commit details
    Browse the repository at this point in the history

  12. actually reading in the VDS

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    3e3b55b View commit details
    Browse the repository at this point in the history

  13. changed reading in vds to inside sites_table function

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    5af3cab View commit details
    Browse the repository at this point in the history

  14. removing new_python_job

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    1123a76 View commit details
    Browse the repository at this point in the history

  15. placing gcs_output_path in correct spot

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    1c40f19 View commit details
    Browse the repository at this point in the history

  16. not filtering decoy's as reference data does not exist in gnomad reso… 

    …urces
    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    1ff0361 View commit details
    Browse the repository at this point in the history

  17. removing .count() calls to speed up testing

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    5ce3162 View commit details
    Browse the repository at this point in the history

  18. correcting checkpoint path

    @michael-harper
    michael-harper committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    3091ea4 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. not using get_qc_mt() as no control of ld_pruning base pair window si… 

    …ze. Filtering according to gnomAD parameters manually after performing hl.variant_qc()
    @michael-harper
    michael-harper committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    61176ff View commit details
    Browse the repository at this point in the history

  2. updating output_path()

    @michael-harper
    michael-harper committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    ad4b407 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/main' into sites-table-generation

    @michael-harper
    michael-harper committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    c589863 View commit details
    Browse the repository at this point in the history

Commits on May 1, 2024

  1. Changing back to v3 parameters and filtering to only exome sites prio… 

    …r to ld-pruning
    @michael-harper
    michael-harper committed May 1, 2024
    Configuration menu
    Copy the full SHA
    82dcbc1 View commit details
    Browse the repository at this point in the history

  2. writing to tmp location for testing purposes

    @michael-harper
    michael-harper committed May 1, 2024
    Configuration menu
    Copy the full SHA
    9a6ef06 View commit details
    Browse the repository at this point in the history

  3. printing out number of variants in pruned table

    @michael-harper
    michael-harper committed May 1, 2024
    Configuration menu
    Copy the full SHA
    9bca163 View commit details
    Browse the repository at this point in the history

  4. Ensuring output path is correct at start of script

    @michael-harper
    michael-harper committed May 1, 2024
    Configuration menu
    Copy the full SHA
    fee913f View commit details
    Browse the repository at this point in the history

Commits on May 2, 2024

  1. downsampling variant sites prior to ld-pruning. Without this step, th… 

    …e exome_pruned_variants had more variants than the pruned_variants sites table that Kat generated with WGS
    @michael-harper
    michael-harper committed May 2, 2024
    Configuration menu
    Copy the full SHA
    c1ea650 View commit details
    Browse the repository at this point in the history

  2. printing out input vds path for clarity

    @michael-harper
    michael-harper committed May 2, 2024
    Configuration menu
    Copy the full SHA
    f8f189c View commit details
    Browse the repository at this point in the history

  3. Changing checkpointing and writing directories to test instead of tes… 

    …t-tmp when building sites_table using the main version of the background_vds
    @michael-harper
    michael-harper committed May 2, 2024
    Configuration menu
    Copy the full SHA
    af18d66 View commit details
    Browse the repository at this point in the history

Commits on May 3, 2024

  1. Running out of storage, high mem workers increase storage of each wor… 

    …ker job
    @michael-harper
    michael-harper committed May 3, 2024
    Configuration menu
    Copy the full SHA
    7a48cc4 View commit details
    Browse the repository at this point in the history

  2. filtering to just exomes instead of down sizing

    @michael-harper
    michael-harper committed May 3, 2024
    Configuration menu
    Copy the full SHA
    ef67347 View commit details
    Browse the repository at this point in the history

Commits on May 6, 2024

  1. Running out of memory. Increasing the driver memory to high, and incr… 

    …easing the number of cores to 4 as it is a driver job that is failing
    @michael-harper
    michael-harper committed May 6, 2024
    Configuration menu
    Copy the full SHA
    8a5f21f View commit details
    Browse the repository at this point in the history

  2. typo in 'highmem'

    @michael-harper
    michael-harper committed May 6, 2024
    Configuration menu
    Copy the full SHA
    8015363 View commit details
    Browse the repository at this point in the history