Running problems with Snakemake 8.x on HILBERT

[svenwillger]

I created on HILBERT a conda exec_env environment with the following packages to be able to test the new Snakemake 8.5.3 version:

name: smk_exec_env
dependencies:
  - Python
  - pip
  - mamba
  - snakemake=8
  - portalocker
  - pandas
  - pytables
  - hdf5
  - pyarrow
  - numpy
  - pyyaml
  - toml
  - semver
  - coincbc

When trying to run the snaketest.smk workflow with the command snakemake -s workflow/snaketests.smk --profile ../prf_MODS_wt_smk8/ --config devmode=True -n with the profile prf_MODS_wt_smk8, which contains the following config.yaml:

cluster-generic-cancel-cmd: qdel
cluster-generic-status-cmd: pbs-status.py
cluster-generic-submit-cmd: >-
  pbs-submit.py ++mkdirs log/cluster_jobs/err,log/cluster_jobs/out
  -e log/cluster_jobs/err/{rule}.{jobid}.stderr
  -o log/cluster_jobs/out/{rule}.{jobid}.stdout
  -N {jobid}_{rule}
  -A {resources.project}
  -l select=1:ncpus={threads}:ngpus={resources.gpus}:mem={resources.mem_mb}M{resources.arch}
  -l walltime={resources.walltime}
  -l bonus={resources.bonus}
conda-frontend: mamba
default-resources:
- project=MODS_wt_smk8
- mem_mb=1024
- walltime="01:59:00"
- gpus=0
- bonus=0
- anchor=""
- arch=""
immediate-submit: false
jobs: 100
jobscript: pbs-jobscript.sh
keep-going: true
keep-incomplete: false
latency-wait: 60
local-cores: 2
max-jobs-per-second: 5
max-status-checks-per-second: 10
nolock: true
rerun-incomplete: true
restart-times: 1
scheduler: ilp
software-deployment-method:
- conda
- apptainer
- env-modules
verbose: false

this is the output:

Using profile ../prf_MODS_wt_smk8/ for setting default command line arguments.
/gpfs/project/projects/medbioinf/projects/cubi/projects/new_smk8_test/repo/workflow/rules/commons/30_settings.smk:209: SyntaxWarning: invalid escape sequence '\-'
  RUN_SUFFIX = RUN_SUFFIX.replace(".", "-").replace("_", "-")
/gpfs/project/projects/medbioinf/projects/cubi/projects/new_smk8_test/repo/workflow/rules/commons/30_settings.smk:212: SyntaxWarning: invalid escape sequence '\-'
  # consecutive hyphens, replace with single one
Traceback (most recent call last):
  File "/gpfs/project/projects/medbioinf/projects/cubi/projects/new_smk8_test/exec_env/lib/python3.12/site-packages/snakemake/cli.py", line 1933, in args_to_api
    dag_api = workflow_api.dag(
              ^^^^^^^^^^^^^^^^^
  File "/gpfs/project/projects/medbioinf/projects/cubi/projects/new_smk8_test/exec_env/lib/python3.12/site-packages/snakemake/api.py", line 328, in dag
    return DAGApi(
           ^^^^^^^
  File "<string>", line 6, in __init__
  File "/gpfs/project/projects/medbioinf/projects/cubi/projects/new_smk8_test/exec_env/lib/python3.12/site-packages/snakemake/api.py", line 438, in __post_init__
    self.workflow_api._workflow.dag_settings = self.dag_settings
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs/project/projects/medbioinf/projects/cubi/projects/new_smk8_test/exec_env/lib/python3.12/site-packages/snakemake/api.py", line 385, in _workflow
    workflow.include(
  File "/gpfs/project/projects/medbioinf/projects/cubi/projects/new_smk8_test/exec_env/lib/python3.12/site-packages/snakemake/workflow.py", line 1386, in include
    exec(compile(code, snakefile.get_path_or_uri(), "exec"), self.globals)
  File "/gpfs/project/projects/medbioinf/projects/cubi/projects/new_smk8_test/repo/workflow/snaketests.smk", line 1, in <module>
    include: "rules/commons/00_commons.smk"
  File "/gpfs/project/projects/medbioinf/projects/cubi/projects/new_smk8_test/exec_env/lib/python3.12/site-packages/snakemake/workflow.py", line 1386, in include
    exec(compile(code, snakefile.get_path_or_uri(), "exec"), self.globals)
  File "/gpfs/project/projects/medbioinf/projects/cubi/projects/new_smk8_test/repo/workflow/rules/commons/00_commons.smk", line 41, in <module>
  File "/gpfs/project/projects/medbioinf/projects/cubi/projects/new_smk8_test/exec_env/lib/python3.12/site-packages/snakemake/workflow.py", line 1386, in include
    exec(compile(code, snakefile.get_path_or_uri(), "exec"), self.globals)
  File "/gpfs/project/projects/medbioinf/projects/cubi/projects/new_smk8_test/repo/workflow/rules/commons/90_staging.smk", line 20, in <module>
    _extract_and_set_dryrun_constant()
  File "/gpfs/project/projects/medbioinf/projects/cubi/projects/new_smk8_test/repo/workflow/rules/commons/40_pyutils.smk", line 585, in _extract_and_set_dryrun_constant
    from snakemake import get_argument_parser as get_smk_cli_parser
ImportError: cannot import name 'get_argument_parser' from 'snakemake' (/gpfs/project/projects/medbioinf/projects/cubi/projects/new_smk8_test/exec_env/lib/python3.12/site-packages/snakemake/__init__.py)