fix conflicts

genomic-medicine-sweden · Oct 13, 2024 · 69917cb · 69917cb
2 parents 54928dd + 9bc110d
commit 69917cb
Show file tree

Hide file tree

Showing 332 changed files with 34,573 additions and 623 deletions.
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -0,0 +1,5 @@
+# These owners will be the default owners for everything in
+# the repo. Unless a later match takes precedence,
+# @global-owner1 and @global-owner2 will be requested for
+# review when someone opens a pull request.
+* @genomic-medicine-sweden/tomte-devs
diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md
@@ -9,6 +9,8 @@ Please use the pre-filled template to save time.
 However, don't be put off by this template - other more general issues and suggestions are welcome!
 Contributions to the code are even more welcome ;)
 
+If you need help using or modifying genomic-medicine-sweden/tomte then the best place to ask is by creating an [issue] (https://github.com/genomic-medicine-sweden/tomte/issues).
+
 ## Contribution workflow
 
 If you'd like to write some code for genomic-medicine-sweden/tomte, the standard workflow is as follows:
@@ -36,14 +38,14 @@ There are typically two types of tests that run:
 
 ### Lint tests
 
-`nf-core` has a [set of guidelines](https://nf-co.re/developers/guidelines) which all pipelines must adhere to.
+`nf-core` has a [set of guidelines](https://nf-co.re/developers/guidelines) which we adhere to even if this pipeline is not formally a part of nf-core.
 To enforce these and ensure that all pipelines stay in sync, we have developed a helper tool which runs checks on the pipeline code. This is in the [nf-core/tools repository](https://github.com/nf-core/tools) and once installed can be run locally with the `nf-core pipelines lint <pipeline-directory>` command.
 
 If any failures or warnings are encountered, please follow the listed URL for more documentation.
 
 ### Pipeline tests
 
-Each `nf-core` pipeline should be set up with a minimal set of test-data.
+This pipeline should be set up with a minimal set of test-data.
 `GitHub Actions` then runs the pipeline on this data to ensure that it exits successfully.
 If there are any failures then the automated tests fail.
 These tests are run both with the latest available version of `Nextflow` and also the minimum required version that is stated in the pipeline code.
@@ -56,6 +58,10 @@ These tests are run both with the latest available version of `Nextflow` and als
 - Fix the bug, and bump version (X.Y.Z+1).
 - A PR should be made on `master` from patch to directly this particular bug.
 
+## Getting help
+
+For further information/help, please consult the [tomte documentation](https://github.com/genomic-medicine-sweden/tomte) and don't hesitate to get in touch on by creating an [issue](https://github.com/genomic-medicine-sweden/tomte/issues).
+
 ## Pipeline contribution conventions
 
 To make the genomic-medicine-sweden/tomte code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -43,6 +43,9 @@ jobs:
             profile: "conda"
           - isMaster: false
             profile: "singularity"
+        parameters:
+          - "-profile test,docker"
+          - "-profile test,docker --skip_subsample_region true --skip_downsample true --skip_build_tracks true --skip_stringtie true --skip_vep true --skip_drop_ae true --skip_drop_as true"
     steps:
       - name: Check out pipeline code
         uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4

diff --git a/.github/workflows/download_pipeline.yml b/.github/workflows/download_pipeline.yml
@@ -89,14 +89,14 @@ jobs:
         env:
           NXF_SINGULARITY_CACHEDIR: ./singularity_container_images
           NXF_SINGULARITY_HOME_MOUNT: true
-        run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) -stub -profile test,singularity --outdir ./results
+        run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }})--skip_drop_as true --skip_drop_ae true -stub -profile test,singularity --outdir ./results
       - name: Run the downloaded pipeline (stub run not supported)
         id: run_pipeline
         if: ${{ job.steps.stub_run_pipeline.status == failure() }}
         env:
           NXF_SINGULARITY_CACHEDIR: ./singularity_container_images
           NXF_SINGULARITY_HOME_MOUNT: true
-        run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) -profile test,singularity --outdir ./results
+        run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) --skip_drop_as true --skip_drop_ae true -profile test,singularity --outdir ./results
 
       - name: Count the downloaded number of container images
         id: count_afterwards

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -3,14 +3,235 @@
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## v2.2.1 - [date]
+## X.X.X - [XXXX-XX-XX]
 
-Initial release of genomic-medicine-sweden/tomte, created with the [nf-core](https://nf-co.re/) template.
+### `Added`
+
+- Functionality to create DROP databases and to add samples to existing ones [#147](https://github.com/genomic-medicine-sweden/tomte/pull/147)
+
+### `Fixed`
+
+### `Parameters`
+
+### `Changed`
+
+- Updated modules ensemblvep/filtervep, ensemblvep/vep [#159](https://github.com/genomic-medicine-sweden/tomte/pull/159)
+- Updated gencode version from 37 to 46 [#159](https://github.com/genomic-medicine-sweden/tomte/pull/159)
+- Updated modules using drop drop_config_runAE, drop_config_runAS, drop_sample_annot, and drop_filter_results [#147](https://github.com/genomic-medicine-sweden/tomte/pull/147)
+
+| Tool                 | Old version | New version |
+| -------------------- | ----------- | ----------- |
+| ensemblvep/filtervep | 110         | 112         |
+| ensemblvep/vep       | 110         | 112         |
+| DROP                 | 1.3.3       | 1.4.0       |
+
+## 2.2.1 - Scrooge [2024-08-28]
+
+### `Fixed`
+
+- After an update, MultiQC was not outputing data for RnaSeqMetrics so an earlier version will be used [#156](https://github.com/genomic-medicine-sweden/tomte/pull/156)
+
+### `Changed`
+
+- Downgraded multiqc version [#156](https://github.com/genomic-medicine-sweden/tomte/pull/156)
+
+| Tool    | Old version | New version |
+| ------- | ----------- | ----------- |
+| multiqc | 1.24.1      | 1.21        |
+
+## 2.2.0 - TioDeNadal [2024-08-27]
+
+### `Added`
+
+- Fasta, gtf, vep cache and plugins can now be downloaded automatically by the pipeline if they are not provided by the user [#149](https://github.com/genomic-medicine-sweden/tomte/pull/149)
+- Added `--gencode_annotation_version`, the version of the gencode reference version to download if fasta or gtf is not provided [#149](https://github.com/genomic-medicine-sweden/tomte/pull/149)
+- Added the possibility to provide `--vep_refs_download`, a comma separated csv determining the vep references that should be downloaded (excluding gnomad ones) alongside with a switch `--skip_download_vep` for the vep reference download in general and `--skip_download_gnomad` for gnomad in particular [#149](https://github.com/genomic-medicine-sweden/tomte/pull/149)
+
+### `Fixed`
+
+- Input to BootstrapAnn is now supplied in a single channel. Previously they were supplied in separate channels, which could cause mix-ups if more than one sample was supplied [#151](https://github.com/genomic-medicine-sweden/tomte/pull/151)
+
+### `Parameters`
+
+| Old parameter | New parameter                  |
+| ------------- | ------------------------------ |
+|               | `--gencode_annotation_version` |
+|               | `--vep_refs_download`          |
+|               | `--skip_download_vep`          |
+|               | `--skip_download_gnomad`       |
+
+> [!NOTE]
+> Parameter has been updated if both old and new parameter information is present.
+> Parameter has been added if just thenew parameter information is present.
+> Parameter has been removed if new parameter information isn't present.
+
+### `Changed`
+
+- Updated modules bcftools/annotate, bcftools/mpileup, bcftools/view, cat/fastq, ensemblvep/filtervep, fastp, fastqc, gatk4/haplotypecaller, gatk4/splitncigarreads, gunzip, multiqc, picard/collectrnaseqmetrics, samtools/index, star/align, star/genomegenerate, stringtie/stringtie, tabix/bgziptabix, tabix/tabix and untar [#153](https://github.com/genomic-medicine-sweden/tomte/pull/153)
+
+| Tool                            | Old version | New version |
+| ------------------------------- | ----------- | ----------- |
+| gunzip                          | 20.04       | 22.04       |
+| multiqc                         | 1.22.3      | 1.24.1      |
+| picard/collectinsertsizemetrics | 3.1.1       | 3.2.0       |
+| tabix/bgziptabix                | 1.19.1      | 1.20        |
+| tabix/tabix                     | 1.19.1      | 1.20        |
+| untar                           | 20.04       | 22.04       |
+
+## 2.1.0 - Elf [2024-06-26]
 
 ### `Added`
 
+- Installed bcftools/norm [#127](https://github.com/genomic-medicine-sweden/tomte/pull/127)
+- Installed bcftools/annotate [#127](https://github.com/genomic-medicine-sweden/tomte/pull/127)
+
 ### `Fixed`
 
+- One line per call on vcf to make vcf suitable for Scout [#127](https://github.com/genomic-medicine-sweden/tomte/pull/127)
+- Added variant caller to vcf to make vcf suitable for Scout [#127](https://github.com/genomic-medicine-sweden/tomte/pull/127)
+- Normalised calls in vcf [#127](https://github.com/genomic-medicine-sweden/tomte/pull/127)
+- Removed regions parameter from ASEReadCounter to obtain ASE for all regions [#129](https://github.com/genomic-medicine-sweden/tomte/pull/129)
+- Updated container for GET_CHROM_SIZES [#132](https://github.com/genomic-medicine-sweden/tomte/pull/132)
+- Updated container for RENAME_FILES [#132](https://github.com/genomic-medicine-sweden/tomte/pull/132)
+- Fixed GATK's ASEReadCounter by adding bcftools norm to avoid having duplicated positions in vcf [#137](https://github.com/genomic-medicine-sweden/tomte/pull/137)
+
+### `Parameters`
+
+### `Changed`
+
+- Updated template to v2.14.1 [#123](https://github.com/genomic-medicine-sweden/tomte/pull/123)
+- Changed DROP output column names to camel case [#132](https://github.com/genomic-medicine-sweden/tomte/pull/132)
+- Updated bcftools view and norm to be able to create index within the actual module [#137](https://github.com/genomic-medicine-sweden/tomte/pull/137)
+- Updated bcftools annot to be able to create index within the actual module [#140](https://github.com/genomic-medicine-sweden/tomte/pull/140)
+- Updated modules bcftools/merge, bcftools/mpileup, bcftools/stats, ensemblvep/vep, fastp, gawk, multiqc, salmon/quant, samtools/faidx, samtools/index and samtools/view [#141](https://github.com/genomic-medicine-sweden/tomte/pull/141)
+- Removed unused modules bcftools/index and tabix/bgzip [#141](https://github.com/genomic-medicine-sweden/tomte/pull/141)
+
+| Tool             | Old version | New version |
+| ---------------- | ----------- | ----------- |
+| bcftools/index   | 1.18        |             |
+| bcftools/merge   | 1.18        | 1.20        |
+| bcftools/mpileup | 1.18        | 1.20        |
+| bcftools/norm    | 1.18        | 1.20        |
+| bcftools/stats   | 1.18        | 1.20        |
+| bcftools/view    | 1.18        | 1.20        |
+| fastp            | 0.23.4      | 0.23.4      |
+| gawk             | 5.1.0       | 5.3.0       |
+| multiqc          | 1.21        | 1.22.3      |
+| salmon/quant     | 1.10.1      | 1.10.1      |
+| samtools/faidx   | 1.19.2      | 1.20        |
+| samtools/index   | 1.19.2      | 1.20        |
+| samtools/view    | 1.19.2      | 1.20        |
+| tabix/bgzip      | 1.19.1      |             |
+
+:::note Version has been updated if both old and new version information is present. Version has been added if just the new version information is present. Version has been removed if new version information isn't present. :::
+
+## 2.0.1 - Grinch [2024-04-25]
+
+### `Added`
+
+### `Fixed`
+
+- Vep annotated research results will be published [#115](https://github.com/genomic-medicine-sweden/tomte/pull/115)
+
+### `Parameters`
+
+## 2.0.0 - Santa [2024-04-19]
+
+### `Added`
+
+- Added automatic tests to test the pipeline with all switches set to false [#100](https://github.com/genomic-medicine-sweden/tomte/pull/100)
+- Added better documentation on subworkflow input [#101](https://github.com/genomic-medicine-sweden/tomte/pull/101)
+- Added option to add extra arguments to DROP aberrant expression and aberrant splicing [#104](https://github.com/genomic-medicine-sweden/tomte/pull/104)
+- Added a function to branch references into compressed/uncompressed [#107](https://github.com/genomic-medicine-sweden/tomte/pull/107)
+- Added nf-core modules gawk and filter vep to create a clinical vcf [#109](https://github.com/genomic-medicine-sweden/tomte/pull/109)
+
+### `Fixed`
+
+- Subsample and downsample switches [#97](https://github.com/genomic-medicine-sweden/tomte/pull/97)
+- Now all reference files come with meta to avoid confusion [#101](https://github.com/genomic-medicine-sweden/tomte/pull/101)
+- GATK4_ASEREADCOUNTER and GATK4_SPLITNCIGARREADS have been updated [#101](https://github.com/genomic-medicine-sweden/tomte/pull/101)
+- Updated GATK4_ASEREADCOUNTER, now bam and vcf will be given as one channel [#103](https://github.com/genomic-medicine-sweden/tomte/pull/103)
+- Prepare reference subworkflow has been reformated and simplified [#105](https://github.com/genomic-medicine-sweden/tomte/pull/105)
+- FastQC have been updated to correctly allocate memory [#106](https://github.com/genomic-medicine-sweden/tomte/pull/106)
+- vep_filters is now extracted from gene_panel_clinical_filter [#109](https://github.com/genomic-medicine-sweden/tomte/pull/109)
+- Updated modules bcftools/stats, ensemblvep/vep, fastp, gatk4/bedtointervallist, samtools/faidx [#110](https://github.com/genomic-medicine-sweden/tomte/pull/110)
+
+### `Parameters`
+
+- Removed `--vep_filters`, it will now be automatically extracted from the `--gene_panel_clinical_filter`[#109](https://github.com/genomic-medicine-sweden/tomte/pull/109)
+  | Old parameter | New parameter |
+  | --------------- | ------------- |
+  | `--vep_filters` | |
+
+- Updated parameter names to make their use easier and more clear, changing the names from `switch` to `skip` and their default value from `true` to `false` [#108](https://github.com/genomic-medicine-sweden/tomte/pull/108)
+
+| Old parameter               | New parameter             |
+| --------------------------- | ------------------------- |
+| `--switch_subsample_region` | `--skip_subsample_region` |
+| `--switch_downsample`       | `--skip_downsample`       |
+| `--switch_build_tracks`     | `--skip_build_tracks`     |
+| `--switch_stringtie`        | `--skip_stringtie`        |
+| `--switch_vep`              | `--skip_vep`              |
+| `--switch_drop_ae`          | `--skip_drop_ae`          |
+| `--switch_drop_as`          | `--skip_drop_as`          |
+
+:::note Parameter has been updated if both old and new parameter information is present. Parameter has been added if just the new parameter information is present. Parameter has been removed if new parameter information isn't present. :::
+
+## 1.1.0 - Rudolph [2024-03-11]
+
+Release of genomic-medicine-sweden/tomte, created with the [nf-core](https://nf-co.re/) template.
+
+### `Added`
+
+- switch_vep, switch_build_tracks and switch_stringtie to make the pipeline more versatile [#61](https://github.com/genomic-medicine-sweden/tomte/pull/61)
+- Updated template and nf-tools [#65](https://github.com/genomic-medicine-sweden/tomte/pull/65)
+- Use `nf-validation` plugin for parameter and samplesheet validation [#66](https://github.com/genomic-medicine-sweden/tomte/pull/66)
+- Installed the nf-core version of ensemblvep/vep module [#67](https://github.com/genomic-medicine-sweden/tomte/pull/67)
+- A new parameter `vep_plugin_files` to supply files required by vep plugins [#67](https://github.com/genomic-medicine-sweden/tomte/pull/67)
+- The possibility of using `igenomes_base` to point to a path where genome specific reference files are placed (fasta, fai, gtf, star_index, salmon_index, subsample_bed) [#76](https://github.com/genomic-medicine-sweden/tomte/pull/76)
+- Merging of case's vcf files [#80](https://github.com/genomic-medicine-sweden/tomte/pull/80)
+- Reference list to MultiQC report [#88](https://github.com/genomic-medicine-sweden/tomte/pull/88)
+- Added module to calculate insert size and added results to MultiQC report [#90](https://github.com/genomic-medicine-sweden/tomte/pull/90)
+
+### `Fixed`
+
+- Renamed the other switches (subsample_region_switch, downsample_switch, run_drop_ae_switch and run_drop_as_switch) so that they all start with switch\* (switch_subsample_region, switch_downsample, switch_drop_ae and switch_drop_as) [#61](https://github.com/genomic-medicine-sweden/tomte/pull/61)
+- Separated modules.config into smaller configs [#61](https://github.com/genomic-medicine-sweden/tomte/pull/61)
+- Missing fasta_fai channel when fai file is given [#63](https://github.com/genomic-medicine-sweden/tomte/pull/63)
+- DROP output file columns, removing duplicate column and adding same ids to both AE and AS [#68](https://github.com/genomic-medicine-sweden/tomte/pull/68)
+- Patch tools update and case ID parsing [#71](https://github.com/genomic-medicine-sweden/tomte/pull/71)
+- Naming of DROP output files [#72](https://github.com/genomic-medicine-sweden/tomte/pull/72)
+- VEP plugin schema to allow for directories [#74](https://github.com/genomic-medicine-sweden/tomte/pull/74)
+- Made params.platform into a channel [#75](https://github.com/genomic-medicine-sweden/tomte/pull/75)
+- Changed name of salmon's quant.nf to include sample id [#78](https://github.com/genomic-medicine-sweden/tomte/pull/78)
+- Shortened name of DROP output files [#79](https://github.com/genomic-medicine-sweden/tomte/pull/79)
+- Merging of vcfs has been moved to after bootstrapAnn [#81](https://github.com/genomic-medicine-sweden/tomte/pull/81)
+- Substituted bgzip and tabix modules by bgzip_tabix module [#85](https://github.com/genomic-medicine-sweden/tomte/pull/85)
+- Updated module input channels in the GATK variant calling subworkflow [#89](https://github.com/genomic-medicine-sweden/tomte/pull/89)
+
 ### `Dependencies`
 
 ### `Deprecated`
+
+## 1.0.0 - Nisse [2023-11-06]
+
+### `Added`
+
+- Trim reads with FASTP
+- Read mapping with STAR
+- Transcript quantification with Salmon
+- Output junction tracks
+- Output bigwig
+- Choice to subsample overrepresented regions with Samtools
+- Choice to downsample number of reads with Samtools
+- Detection of aberrant expression with DROP
+- Detection of aberrant splicing with DROP
+- Filter aberrant expression and aberrant splicing results
+- Guided transcript assembly with StringTie
+- Filtering results of guided transcript assembly with GffCompare
+- SNVs calling with GATK or BCFtools Mpileups
+- Allele Specific Read Counter with ASEReadCounter
+- Assess allelic imbalance with BootstrapAnn
+- Annotation with VEP
+- Alignment QC with Picard CollectRnaSeqMetrics
+- Present QCs with MultiQC