Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

final tarball for built compat layer not created correctly by bot/build.sh #189

Open
boegel opened this issue Jun 14, 2023 · 1 comment
Open
Labels
bug Something isn't working

Comments

@boegel
Copy link
Contributor

boegel commented Jun 14, 2023

As shown in #188, building the 2023.06 compat layer autonomously worked, but creating the tarball for ingestion failed because of a bug at the end of the bot/build.sh script.

Relevant part of output of the Slurm build job:

PLAY RECAP *********************************************************************
localhost                  : ok=63   changed=38   unreachable=0    failed=0    skipped=3    rescued=0    ignored=1

Left container; tar'ing up /tmp/bot/EESSI/eessi.BM69QGLjmn for future inspection
...
created tarball '/mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.06/pr_188/event_bf958ee0-0a82-11ee-93ec-8e0bfaa03c26/run_001/linux_aarch64_generic/eessi-2023.06-compat/job_4940_1686744594.tgz'
To resume work add '--resume /tmp/bot/EESSI/eessi.BM69QGLjmn'
Directory //cvmfs/pilot.eessi-hpc.org/versions/2023.06 was not created, not creating tarball.

The job_*.tgz mentioned not only includes /cvmfs/..., but also /tmp and apptainer_cache, so it can't be used for ingestion.

@boegel boegel added the bug Something isn't working label Jun 14, 2023
@trz42
Copy link
Contributor

trz42 commented Jun 14, 2023

Left container; tar'ing up /tmp/bot/EESSI/eessi.BM69QGLjmn for future inspection
...
created tarball '/mnt/shared/home/bot/eessi-bot-software-layer/jobs/2023.06/pr_188/event_bf958ee0-0a82-11ee-93ec-8e0bfaa03c26/run_001/linux_aarch64_generic/eessi-2023.06-compat/job_4940_1686744594.tgz'
To resume work add '--resume /tmp/bot/EESSI/eessi.BM69QGLjmn'
Directory //cvmfs/pilot.eessi-hpc.org/versions/2023.06 was not created, not creating tarball.


The `job_*.tgz` mentioned not only includes `/cvmfs/...`, but also `/tmp` and `apptainer_cache`, so it can't be used for ingestion.

As the first line states, this tarball is intended for future inspection not for ingestion. It's essentially the complete temporary directory used during the job or more precisely what install_compatibility_layer.sh uses as base folder for temporary data. It includes cvmfs, home, tmp and possibly container_cachedir.

What's missing in the bot/build.sh script is assigning a value to eessi_tmpdir. Adding a line such as

eessi_tmpdir=${STORAGE}

before

# create tarball -> should go into a separate script when this is supported by the bot
target_tgz=eessi-${eessi_version}-compat-linux-${eessi_arch}-$(date +%s).tar.gz
if [ -d ${eessi_tmp}/${tar_topdir}/${eessi_version} ]; then
echo ">> Creating tarball ${target_tgz} from ${eessi_tmp}/${tar_topdir}..."
tar cfvz ${target_tgz} -C ${eessi_tmp}/${tar_topdir} ${eessi_version}/compat/${eessi_os}/${eessi_arch}
echo ${target_tgz} created!
else
echo "Directory ${eessi_tmp}/${tar_topdir}/${eessi_version} was not created, not creating tarball."
exit 1
fi
would probably fix it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants