Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add nf-test to CI #86

Open
wants to merge 10 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 109 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,43 +4,143 @@ on:
push:
branches:
- dev
- master
pull_request:
release:
types: [published]

env:
NXF_ANSI_LOG: false

NFTEST_VER: "0.8.1"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can use 0.9

# cancel if a newer run is started
concurrency:
group: "${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}"
cancel-in-progress: true

jobs:
test:
name: Run pipeline with test data
nftest-changes:
name: Check for changes (nf-test)
runs-on: ubuntu-latest
outputs:
tags: ${{ steps.filter.outputs.changes }}

steps:
- uses: actions/checkout@v4

- name: Combine all tags.yml files
id: get_tags
run: find . -name "tags.yml" -not -path "./.github/*" -exec cat {} + > .github/tags.yml

- name: debug
run: cat .github/tags.yml

- uses: frouioui/paths-filter@main
id: filter
with:
filters: ".github/tags.yml"
token: ""

nftest:
name: ${{ matrix.tags }} ${{ matrix.profile }} NF ${{ matrix.NXF_VER }}
# Only run on push if this is the nf-core dev branch (merged PRs)
if: "${{ github.event_name != 'push' || (github.event_name == 'push' && github.repository == 'nf-core/variantbenchmarking') }}"
if: needs.nftest-changes.outputs.tags != '[]'
runs-on: ubuntu-latest
needs: nftest-changes

strategy:
fail-fast: false
matrix:
tags: ["${{ fromJson(needs.nftest-changes.outputs.tags) }}"]
profile: ["docker"]
# profile: ["docker", "singularity", "conda"]
NXF_VER:
- "23.04.0"
- "latest-everything"
include:
- tags: "pipeline_variantbenchmarking"
profile: "docker"

steps:
- name: Check out pipeline code
uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b # v4

- uses: actions/setup-java@v3
with:
distribution: "temurin"
java-version: "17"

- name: Install Nextflow
uses: nf-core/setup-nextflow@v2
with:
version: "${{ matrix.NXF_VER }}"

- name: Cache nf-test installation
id: cache-software
uses: actions/cache@v3
with:
path: |
/usr/local/bin/nf-test
/home/runner/.nf-test/nf-test.jar
key: ${{ runner.os }}-${{ env.NFTEST_VER }}-nftest

- name: Install nf-test
if: steps.cache-software.outputs.cache-hit != 'true'
run: |
wget -qO- https://code.askimed.com/install/nf-test | bash
sudo mv nf-test /usr/local/bin/

- name: Setup apptainer
if: matrix.profile == 'singularity'
uses: eWaterCycle/setup-apptainer@main

- name: Set up Singularity
if: matrix.profile == 'singularity'
run: |
mkdir -p $NXF_SINGULARITY_CACHEDIR
mkdir -p $NXF_SINGULARITY_LIBRARYDIR

- name: Set up miniconda
uses: conda-incubator/setup-miniconda@v2
with:
miniconda-version: "latest"
auto-update-conda: true
channels: conda-forge,bioconda,defaults
python-version: ${{ matrix.python-version }}

- name: Conda setup
run: |
conda clean -a
conda install -n base conda-libmamba-solver
conda config --set solver libmamba
echo $(realpath $CONDA)/condabin >> $GITHUB_PATH
echo $(realpath python) >> $GITHUB_PATH

- name: Disk space cleanup
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1

- name: Run pipeline with test data for germline analysis
# TODO nf-core: You can customise CI pipeline run tests as required
# For example: adding multiple test runs with different parameters
# Remember that you can parallelise this by using strategy.matrix
- name: Run nf-test
run: |
nf-test test tests/main.nf.test \
--profile=${{ matrix.profile }} \
--tag ${{ matrix.tags }} \
--tap=test.tap \
--verbose

confirm-pass:
runs-on: ubuntu-latest
needs:
- nftest
if: always()
steps:
- name: All tests ok
if: ${{ success() || !contains(needs.*.result, 'failure') }}
run: exit 0
- name: One or more tests failed
if: ${{ contains(needs.*.result, 'failure') }}
run: exit 1

- name: debug-print
if: always()
run: |
nextflow run ${GITHUB_WORKSPACE} -profile test,docker --outdir ./results
echo "toJSON(needs) = ${{ toJSON(needs) }}"
echo "toJSON(needs.*.result) = ${{ toJSON(needs.*.result) }}"
2 changes: 2 additions & 0 deletions .nf-core.yml
Original file line number Diff line number Diff line change
@@ -1,2 +1,4 @@
repository_type: pipeline
nf_core_version: "2.14.1"
lint:
actions_ci: false
4 changes: 4 additions & 0 deletions tests/config/pytesttags.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# default
default:
- "**"
# default_extended
66 changes: 64 additions & 2 deletions tests/main.nf.test
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@ nextflow_pipeline {

name "Test pipeline"
script "../main.nf"
tag "PIPELINE"
tag "pipeline"
tag "pipeline_variantbenchmarking"

test("Run with profile test") {

Expand All @@ -16,7 +17,7 @@ nextflow_pipeline {
outdir = "$outputDir"
genome = 'GRCh38'
analysis = 'germline'
method = 'happy,truvari,svanalyzer,wittyer,rtgtools' //
method = 'happy,truvari,svanalyzer,wittyer,rtgtools'
preprocess = "normalization,deduplication,prepy"
sv_standardization = "svync,homogenize"
include_expression = 'FILTER="."'
Expand All @@ -29,6 +30,67 @@ nextflow_pipeline {
}
}

then {
assertAll(
{ assert workflow.success }
)
}
}
test("Run with profile test somatic") {

when {
params {
max_cpus = 2
max_memory = '6.GB'
max_time = '6.h'
input = 'https://raw.githubusercontent.com/kubranarci/benchmark_datasets/main/sarek_full_test_somatic_v3.4.2/samplesheet_somatic.csv'
outdir = 'results'
genome = 'GRCh38'
analysis = 'somatic'
sample = "SEQC2"
method = 'sompy,truvari'
preprocess = "filter_contigs"
truth_snv = "https://raw.githubusercontent.com/kubranarci/benchmark_datasets/main/somatic/truth/hg38/sSNV_truth_set_v1.0.chr21.vcf.gz"
high_conf_snv = "https://raw.githubusercontent.com/kubranarci/benchmark_datasets/main/somatic/truth/hg38/high-confidence_sSNV_in_HC_regions_v1.2.chr21.vcf.gz"
truth_indel = "https://raw.githubusercontent.com/kubranarci/benchmark_datasets/main/somatic/truth/hg38/sINDEL_truth_set_v1.0.chr21.vcf.gz"
high_conf_indel = "https://raw.githubusercontent.com/kubranarci/benchmark_datasets/main/somatic/truth/hg38/high-confidence_sINDEL_in_HC_regions_v1.2.chr21.vcf.gz"
truth_sv = "https://raw.githubusercontent.com/kubranarci/benchmark_datasets/main/somatic/truth/hg38/13059_2022_2816_MOESM4_ESM.vcf.gz"
itruth_ignore = true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you just need outdir as the rest is in your test profile

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then, I should add config as

config "../conf/test.config"

?

}
}

then {
assertAll(
{ assert workflow.success }
)
}
}
test("Run with profile test liftover") {

when {
params {
max_cpus = 2
max_memory = '6.GB'
max_time = '6.h'
input = 'https://raw.githubusercontent.com/kubranarci/benchmark_datasets/main/SV_testdata/samplesheet_HG002_hg38.csv'
outdir = 'results'
genome = 'GRCh38'
analysis = 'germline'
sample = "HG002"
method = 'truvari,svanalyzer,rtgtools,happy'
preprocess = "normalization,deduplication,filter_contigs"
min_sv_size = 30
truth_small = "https://raw.githubusercontent.com/kubranarci/benchmark_datasets/main/SV_testdata/hg37/truth/HG002_GRCh37_1_22_v4.2.1_highconf.chr21.vcf.gz"
high_conf_small = "https://raw.githubusercontent.com/kubranarci/benchmark_datasets/main/SV_testdata/hg37/truth/HG002_GRCh37_1_22_v4.2.1_highconf.bed"
truth_sv = "https://raw.githubusercontent.com/kubranarci/benchmark_datasets/main/SV_testdata/hg37/truth/HG002_SVs_Tier1_v0.6.chr21.vcf.gz"
high_conf_sv = "https://raw.githubusercontent.com/kubranarci/benchmark_datasets/main/SV_testdata/hg37/truth/HG002_SVs_Tier1_v0.6.chr21.bed"
itruth_ignore = true
chain = "https://raw.githubusercontent.com/broadinstitute/gatk/master/scripts/funcotator/data_sources/gnomAD/b37ToHg38.over.chain"
rename_chr = "https://raw.githubusercontent.com/kubranarci/benchmark_datasets/main/SV_testdata/grch37_grch38.txt"
liftover = true
}
}

then {
assertAll(
{ assert workflow.success }
Expand Down
5 changes: 5 additions & 0 deletions tests/tags.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pipeline_variantbenchmarking:
- "**.nf"
- "**.config"
- "**.nf.test"
- "**.json"
Loading