Conda CI Failure Late November 2021 in packages linking OpenGL directly or transitively (i.e. via qt)

[traversaro]

The error

This issue is an explanation for the failure that affected several repos that we manage at IIT and have a conda-forge-based CI, and that experience failures like:

2021-11-23T02:37:07.3076581Z [1028/1168] Linking CXX executable bin/yarpview
2021-11-23T02:37:07.3077160Z FAILED: bin/yarpview 
2021-11-23T02:37:07.3089693Z : && /usr/share/miniconda3/envs/test/bin/x86_64-conda-linux-gnu-c++ -Wall -Wextra -Wsign-compare -Wpointer-arith -Winit-self -Wnon-virtual-dtor -Wcast-align -Wunused -Wunused-but-set-variable -Wvla -Wmissing-include-dirs -Wlogical-op -Wreorder -Wsizeof-pointer-memaccess -Woverloaded-virtual -Wundef -Wredundant-decls -Wunknown-pragmas -Wunused-result -Wc++17-compat -Wignored-attributes -Wdangling-else -Wmisleading-indentation -Wtautological-compare -Wsuggest-override -Wmaybe-uninitialized  -Wno-unused-parameter  -Wdeprecated-declarations -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /usr/share/miniconda3/envs/test/include -O3 -DNDEBUG -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/usr/share/miniconda3/envs/test/lib -Wl,-rpath-link,/usr/share/miniconda3/envs/test/lib -L/usr/share/miniconda3/envs/test/lib src/yarpview/src/CMakeFiles/yarpview.dir/yarpview_autogen/mocs_compilation.cpp.o src/yarpview/src/CMakeFiles/yarpview.dir/main.cpp.o src/yarpview/src/CMakeFiles/yarpview.dir/qtquick2applicationviewer/qtquick2applicationviewer.cpp.o src/yarpview/src/CMakeFiles/yarpview.dir/qrc_res.cpp.o -o bin/yarpview  -Wl,-rpath,:::::::::::::::::::::::::::::::::::  /usr/share/miniconda3/envs/test/lib/libQt5Widgets.so.5.12.9  /usr/share/miniconda3/envs/test/lib/libQt5Quick.so.5.12.9  /usr/share/miniconda3/envs/test/lib/libQt5Qml.so.5.12.9  /usr/share/miniconda3/envs/test/lib/libQt5Network.so.5.12.9  /usr/share/miniconda3/envs/test/lib/libQt5Gui.so.5.12.9  /usr/share/miniconda3/envs/test/lib/libQt5Core.so.5.12.9 && :
2021-11-23T02:37:07.3100664Z /usr/share/miniconda3/envs/test/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: warning: libxshmfence.so.1, needed by /usr/share/miniconda3/envs/test/lib/libGL.so.1, not found (try using -rpath or -rpath-link)
2021-11-23T02:37:07.3103024Z /usr/share/miniconda3/envs/test/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/share/miniconda3/envs/test/lib/libGL.so.1: undefined reference to `xshmfence_alloc_shm'
2021-11-23T02:37:07.3105156Z /usr/share/miniconda3/envs/test/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/share/miniconda3/envs/test/lib/libGL.so.1: undefined reference to `xshmfence_map_shm'
2021-11-23T02:37:07.3107453Z /usr/share/miniconda3/envs/test/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/share/miniconda3/envs/test/lib/libGL.so.1: undefined reference to `xshmfence_reset'
2021-11-23T02:37:07.3109690Z /usr/share/miniconda3/envs/test/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/share/miniconda3/envs/test/lib/libGL.so.1: undefined reference to `xshmfence_unmap_shm'
2021-11-23T02:37:07.3111961Z /usr/share/miniconda3/envs/test/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/share/miniconda3/envs/test/lib/libGL.so.1: undefined reference to `xshmfence_trigger'
2021-11-23T02:37:07.3114107Z /usr/share/miniconda3/envs/test/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/share/miniconda3/envs/test/lib/libGL.so.1: undefined reference to `memcpy@GLIBC_2.14'
2021-11-23T02:37:07.3116227Z /usr/share/miniconda3/envs/test/bin/../lib/gcc/x86_64-conda-linux-gnu/9.4.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/share/miniconda3/envs/test/lib/libGL.so.1: undefined reference to `xshmfence_await'
2021-11-23T02:37:07.3117454Z collect2: error: ld returned 1 exit status
2021-11-23T02:37:07.3118475Z [1029/1168] Building CXX object src/yarpscope/plugin/CMakeFiles/QtYARPScopePlugin.dir/portreader.cpp.o
2021-11-23T02:37:07.3120063Z [1030/1168] Building CXX object src/yarpscope/plugin/CMakeFiles/QtYARPScopePlugin.dir/QtYARPScopePlugin_autogen/mocs_compilation.cpp.o
2021-11-23T02:37:07.3121237Z ninja: build stopped: subcommand failed.
2021-11-23T02:37:07.3121897Z ninja: build stopped: subcommand failed.

Long and boring explanation

This issue is an explanation of what is happening and a way to indicate possible solutions.
This specific problem is caused by the fact that /usr/share/miniconda3/envs/test/lib/libGL.so.1 is missing the memcpy@GLIBC_2.14 (so a memcpy function provided by glibc 2.14) and a few functions provided libxshmfence.so.1. This was not happening until a few days ago, because the /usr/share/miniconda3/envs/test/lib/libGL.so.1 library was provided by mesalib version 18, that was now updated to version 21 and to require glibc 2.17 and the cdt packages provided by centos7 (see https://conda-forge.org/docs/maintainer/knowledge_base.html#core-dependency-tree-packages-cdts and #477 for what a cdt package is). So, this was triggered by conda-forge/mesalib-feedstock#34 .

However, in theory the libLG.so.1 library should be provided by a cdt package (in particular mesa-libgl-cos6-x86_64, or cos7 if you are targeting glib 2.17), so there is no need to install mesalib at all. So indeed a solution (S1) is to stop installing mesalib.
However, if you install gazebo, mesalib is going to be installed anyway as of Nov 2021 due to this dependency chain:

mesalib <-- libva <-- ffmpeg <-- gazebo

So if you install ``ffmpegorgazebo`, the easiest solution is to switch to `cos7` cdt packages (required by the latest mesalib build) and also install the `libxshmfence-cos7-x86_64 libxshmfence-devel-cos7-x86_64` cdt packages. This is mentioned as S2 in the next section.

TL;DR: Just the solutions

S1: Do not install mesalib

If you do not depend or use gazebo or ffmpeg, just do not install mesalib and install instead mesa-libgl-cos6-x86_64.

S2: Use cos7 packages instead of cos6 package

If you depend or use gazebo or ffmpeg, instead of installing:

libselinux-cos6-x86_64 libxau-cos6-x86_64 libxcb-cos6-x86_64 libxdamage-cos6-x86_64 libxext-cos6-x86_64 libxfixes-cos6-x86_64 libxxf86vm-cos6-x86_64 mesalib mesa-libgl-cos6-x86_64 mesa-libgl-devel-cos6-x86_64

install:

libselinux-cos7-x86_64 libxau-cos7-x86_64 libxcb-cos7-x86_64 libxdamage-cos7-x86_64 libxext-cos7-x86_64 libxfixes-cos7-x86_64 libxxf86vm-cos7-x86_64 mesa-libgl-cos7-x86_64 mesa-libgl-devel-cos7-x86_64 libxshmfence-cos7-x86_64 libxshmfence-devel-cos7-x86_64

The only downside of this is that the compiled binaries will not run in distro release before 2014, but I do not think this is really a problem for any user of robotology-superbuild (if instead it is, please comment!). This is the solution adopted in #926 .

[traversaro]

fyi @Tobias-Fischer @wolfv I imagine that this may affect also Robostack packages that depend on Gazebo or in any indirect way to ffmpeg (for example via ignition-common).

[traversaro]

Note that this is not clear to me why libva has a run dependency on mesalib instead of using cdt packages, but perhaps there is a reason. I guess libva uses also other mesa library that are not just the OpenGL API.

[traversaro]

Related PRs:

• In conda deps do not install mesalib, switch to cos7 and switch back to YARP master in Unstable branches #926
• Fix conda Linux compilation by switching to cos7 cdt packages icub-basic-demos#36
• Fix conda ci by not installing mesalib idyntree#948
• Fix Conda Linux CI compilation by switching to cos7 cdt packages gazebo-yarp-plugins#590

[traversaro]

Closing as all the related PRs have been fixed.

[traversaro]

xref related conda-forge issue: conda-forge/conda-forge.github.io#1436 .