multiqc dies with large number of samples

[frenkiboy]

We used the pipeline for the analysis of ~300 smart seq samples. multiqc died with the following error:
raceback (most recent call last):
File "/gnu/store/crsh6vl6a2pwiad9a9vcd7gyq0wp83xi-multiqc-1.5/bin/.multiqc-real", line 767, in
multiqc()
File "/gnu/store/2h1kcjw1r1306chd45452kwqzq2xb001-python-click-6.7/lib/python3.6/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/gnu/store/2h1kcjw1r1306chd45452kwqzq2xb001-python-click-6.7/lib/python3.6/site-packages/click/core.py", line 676, in main
_verify_python3_env()
File "/gnu/store/2h1kcjw1r1306chd45452kwqzq2xb001-python-click-6.7/lib/python3.6/site-packages/click/_unicodefun.py", line 118, in _verify_python3_env
'for mitigation steps.' + extra)
RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult http://click.pocoo.org/python3/for mitigation steps.

Additional information: on this system no suitable UTF-8
locales were discovered. This most likely requires resolving
by reconfiguring the locale system.

Click discovered that you exported a UTF-8 locale
but the locale system could not pick up from it because
it does not exist. The exported locale is "en_US.UTF-8" but it
is not supported
Traceback (most recent call last):
File "/gnu/store/crsh6vl6a2pwiad9a9vcd7gyq0wp83xi-multiqc-1.5/bin/.multiqc-real", line 767, in
multiqc()
File "/gnu/store/2h1kcjw1r1306chd45452kwqzq2xb001-python-click-6.7/lib/python3.6/site-packages/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/gnu/store/2h1kcjw1r1306chd45452kwqzq2xb001-python-click-6.7/lib/python3.6/site-packages/click/core.py", line 676, in main
_verify_python3_env()
File "/gnu/store/2h1kcjw1r1306chd45452kwqzq2xb001-python-click-6.7/lib/python3.6/site-packages/click/_unicodefun.py", line 118, in _verify_python3_env
'for mitigation steps.' + extra)
RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult http://click.pocoo.org/python3/for mitigation steps.

Additional information: on this system no suitable UTF-8
locales were discovered. This most likely requires resolving
by reconfiguring the locale system.

Click discovered that you exported a UTF-8 locale
but the locale system could not pick up from it because
it does not exist. The exported locale is "en_US.UTF-8" but it
is not supported

[alexg9010]

Have a look here: BIMSBbioinfo/pigx_chipseq#99

[rekado]

Alex is right. You need to set GUIX_LOCPATH so that the glibc can find its locale data (provided by either glibc-locales or the smaller glibc-utf8-locales package). This is unfortunately not a normal dependency, so it is not declared.

The locales really belong to the user's environment setup. Since multiqc fails without proper locale definition we should validate the user's environment before starting the pipeline. This can be as easy as running setlocale to check if locales can be set; if that fails we should bail out with a meaningful error message.

(This is a consequence of supporting different variants of glibc itself; since glibc and its locale data files are not compatible across different glibc versions we must use GUIX_LOCPATH. Upstream glibc honours the LOCPATH variable instead.)

[borauyar]

@rekado should this be closed? This is more of a guix locale dependency issue.

[rekado]

Bora Uyar <notifications@github.com> writes:

@rekado should this be closed? This is more of a guix locale dependency issue.

I think it’s really a validation issue. We should abort early if the user-requested locale cannot be set. This problem could easily happen even if locale data are installed but the user requests an invalid locale (as has happened in the past with some misconfigured SSH clients).

…

-- Ricardo

[borauyar]

@rekado okay, how do we make the pipeline check it before starting the run?