-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
mhcquant fails possibly due to ms2pip #336
Comments
The folder is both read and writable
|
Apologies for the late response here. Additional question: |
The files are both read and write. I enabled the read/write all.
The error could also be due to the high resources needed by
https://github.com/nf-core/mhcquant/blob/master/conf/base.config
Is there a way to provide a config file which ask for lesser resources? |
Yes you can overwrite this by providing a custom config file along nextflows
For Comet there is also an option to split the database search into batches using |
I tried it, however, I still see the same issue
I do not think it is resource related issue. |
The memory usage displayed in the log here
Is not correct. Can you try playing around with memory? Comet definitely needs more than 2 GB. I would give it as much as you can. Also consider using additional resource flags. I am not sure if the custom config actually overwrites the ressourcen limits of nextflow.config. |
|
Very strange. I am also confused why the test run crashes at Comet and your run
did run through CometAdapter 🤔 |
I just had to run the command again with |
Can you upload the full |
|
The nextflow.log you posted stopped at cometadapter again 🤔 . Can you do a non-resume run test run again and maybe give more cpus, maybe max(cpus of your instance) -1 via the |
I added a custom config file. It fails at Comet but different error this time.
|
Description of the bug
Hi,
I am trying to run mhcquant on a SLURM computing cluster. Upon executing the
nextflow run nf-core/mhcquant -profile singularity
I see the error, I have created/home/shubham/.ms2pip
even then I am not able to run it successfully. What could be the issue?Command used and terminal output
(base) [shubham@cfx22885s-srcf-d11-17-west hlaPipeline]$ nextflow run nf-core/mhcquant -profile singularity --input 'samplesheet.tsv' --outdir results --fasta protein.fasta --ms2pip_model timsTOF2023 --ms2pip_model_dir /home/shubham/.ms2pip -resume
N E X T F L O W ~ version 24.04.4
Launching
https://github.com/nf-core/mhcquant
[determined_sax] DSL2 - revision: 1b30692 [master]WARN: Nextflow self-contained distribution allows only core plugins -- User config plugins will be ignored: nf-validation@1.1.3
WARN: Access to undefined parameter
monochromeLogs
-- Initialise it to a default value eg.params.monochromeLogs = some_value
|\ | |__ __ /
/ \ |__) |__ } { | \| | \__, \__/ | \ |___ \
-.,--,
.,._,'nf-core/mhcquant v2.6.0-g1b30692
Core Nextflow options
revision : master
runName : determined_sax
containerEngine : singularity
launchDir : /oak/shubham/hlaPipeline
workDir : /oak/shubham/hlaPipeline/work
projectDir : /home/shubham/.nextflow/assets/nf-core/mhcquant
userName : shubham
profile : singularity
configFiles :
Input/output options
input : samplesheet.tsv
outdir : results
Database Options
fasta : protein.fasta
Database Search Settings
digest_mass_range : 800:2500
prec_charge : 2:3
fixed_mods :
default_params_file_comet:
Rescoring settings
ms2pip_model : timsTOF2023
ms2pip_model_dir : /home/shubham/.ms2pip
!! Only displaying parameters that differ from the pipeline defaults !!
If you use nf-core/mhcquant for your analysis please cite:
The pipeline
https://doi.org/10.1021/acs.jproteome.9b00313
The nf-core framework
https://doi.org/10.1038/s41587-020-0439-x
Software dependencies
https://github.com/nf-core/mhcquant/blob/master/CITATIONS.md
executor > local (1)
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:THERMORAWFILEPARSER -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNTAR -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNZIP -
executor > local (1)
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:THERMORAWFILEPARSER -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNTAR -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNZIP -
executor > local (1)
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:THERMORAWFILEPARSER -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNTAR -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNZIP -
[ad/96526a] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:TDF2MZML (2) [100%] 2 of 2, cached: 2 ✔
executor > local (1)
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:THERMORAWFILEPARSER -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNTAR -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:UNZIP -
[ad/96526a] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:TDF2MZML (2) [100%] 2 of 2, cached: 2 ✔
[- ] process > NFCORE_MHCQUANT:MHCQUANT:PREPARE_SPECTRA:GUNZIP -
[dc/131eb0] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_DECOYDATABASE (protein) [100%] 1 of 1, cached: 1 ✔
[d5/48696c] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_COMETADAPTER (2) [100%] 2 of 2, cached: 2 ✔
[64/bfa4c8] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_PEPTIDEINDEXER (1) [100%] 2 of 2, cached: 2 ✔
[22/9de26a] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_IDMERGER (tumor_treated) [100%] 1 of 1, cached: 1 ✔
[68/26238c] process > NFCORE_MHCQUANT:MHCQUANT:MS2RESCORE (tumor_treated) [100%] 1 of 1, failed: 1 ✘
[- ] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_PSMFEATUREEXTRACTOR -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_PERCOLATORADAPTER -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_IDFILTER_Q_VALUE -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_TEXTEXPORTER -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:OPENMS_MZTABEXPORTER -
[- ] process > NFCORE_MHCQUANT:MHCQUANT:MULTIQC [ 0%] 0 of 1
Execution cancelled -- Finishing pending tasks before exit
-[nf-core/mhcquant] Pipeline completed with errors-
ERROR ~ Error executing process > 'NFCORE_MHCQUANT:MHCQUANT:MS2RESCORE (tumor_treated)'
Caused by:
Process
NFCORE_MHCQUANT:MHCQUANT:MS2RESCORE (tumor_treated)
terminated with an error exit status (1)Command executed:
ms2rescore_cli.py
--psm_file tumor_treated.idXML
--spectrum_path .
--output_path tumor_treated_ms2rescore.idXML
--processes 12
--ms2_tolerance 0.02 --ms2pip_model timsTOF2023 --ms2pip_model_dir /home/shubham/.ms2pip --rescoring_engine percolator --feature_generators deeplc,ms2pip
cat <<-END_VERSIONS > versions.yml$(echo $ (ms2rescore --version 2>&1) | grep -oP 'MS²Rescore (v\K[^\)]+' ))
"NFCORE_MHCQUANT:MHCQUANT:MS2RESCORE":
MS²Rescore:
END_VERSIONS
Command exit status:
1
Command output:
Warning: OPENMS_DATA_PATH environment variable not found and no share directory was installed. Some functionality might not work as expected.
Command error:
/scg/apps/modules/lmod/lmod/init/bash: line 16: _lmod_vx: unbound variable
INFO: Converting SIF file to temporary sandbox...
Matplotlib created a temporary cache directory at /tmp/matplotlib-4afs_8gr because the default path (/home/shubham/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
Fontconfig error: No writable cache directories
2024-10-19 11:18:31,311 INFO Percolator rescoring engine has been specified. Use the idXML containing rescoring features and run Percolator in a separate step.
2024-10-19 11:18:31,311 INFO MS²Rescore config:
2024-10-19 11:18:31,311 INFO {'$schema': './config_schema.json', 'ms2rescore': {'feature_generators': {'ms2pip': {'model': 'timsTOF2023', 'ms2_tolerance': 0.02, 'model_dir': '/home/shubham/.ms2pip'}, 'deeplc': {'deeplc_retrain': False, 'calibration_set_size': 0.15}}, 'rescoring_engine': {}, 'config_file': None, 'psm_file': 'tumor_treated.idXML', 'psm_file_type': 'infer', 'psm_reader_kwargs': {}, 'spectrum_path': '.', 'output_path': 'tumor_treated_ms2rescore.idXML', 'log_level': 'info', 'id_decoy_pattern': '^DECOY', 'psm_id_pattern': None, 'spectrum_id_pattern': None, 'lower_score_is_better': True, 'modification_mapping': {}, 'fixed_modifications': {}, 'processes': 12, 'rename_to_usi': False, 'fasta_file': None, 'write_report': True, 'ms2pip_model_dir': '/home/shubham/.ms2pip'}}
2024-10-19 11:18:50,520 INFO Found 90370 PSMs, of which 48.59% are decoys.
2024-10-19 11:18:51,736 WARNING Non-mapped modifications found: {'Oxidation'}
This can be ignored if Unimod modification label
2024-10-19 11:18:52,292 INFO Found 625 identified PSMs at 1% FDR before rescoring.
2024-10-19 11:18:52,407 INFO Adding MS²PIP-derived features to PSMs.
2024-10-19 11:18:52,552 INFO Running MS²PIP for PSMs from run (1/2)
20240520_HLAI_067_DDA_30min_200ms_7PASEF_Slot1-13_1_1813
...Warning: OPENMS_DATA_PATH environment variable not found and no share directory was installed. Some functionality might not work as expected.
Traceback (most recent call last):
File "/home/shubham/.nextflow/assets/nf-core/mhcquant/bin/ms2rescore_cli.py", line 177, in
sys.exit(main())
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/shubham/.nextflow/assets/nf-core/mhcquant/bin/ms2rescore_cli.py", line 173, in main
rescore_idxml(kwargs["psm_file"], kwargs["output_path"], config)
File "/home/shubham/.nextflow/assets/nf-core/mhcquant/bin/ms2rescore_cli.py", line 82, in rescore_idxml
rescore(config, psm_list)
File "/usr/local/lib/python3.10/site-packages/ms2rescore/core.py", line 80, in rescore
fgen.add_features(psm_list)
File "/usr/local/lib/python3.10/site-packages/ms2rescore/feature_generators/ms2pip.py", line 190, in add_features
ms2pip_results = correlate(
File "/usr/local/lib/python3.10/site-packages/ms2pip/core.py", line 178, in correlate
ms2pip_parallelized = _Parallelized(
File "/usr/local/lib/python3.10/site-packages/ms2pip/core.py", line 383, in init
validate_requested_xgb_model(
File "/usr/local/lib/python3.10/site-packages/ms2pip/_utils/xgb_models.py", line 21, in validate_requested_xgb_model
_download_model(model_file, xgboost_model_hashes[model_file], model_dir)
File "/usr/local/lib/python3.10/site-packages/ms2pip/_utils/xgb_models.py", line 94, in _download_model
os.mkdir(model_dir)
OSError: [Errno 30] Read-only file system: '/home/shubham/.ms2pip'
INFO: Cleaning up image...
Work dir:
/oak/shubham/hlaPipeline/work/68/26238c11b1b4af983e2e710adc8772
Tip: you can replicate the issue by changing to the process work dir and entering the command
bash .command.run
-- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting
-- Check '.nextflow.log' file for details
Relevant files
No response
System information
nextflow/24.04.4
Executor Slurm
Engine Singularity
The text was updated successfully, but these errors were encountered: