Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rpm-ostree OCI-TA Task variant #1502

Merged
merged 2 commits into from
Oct 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions task/rpm-ostree-oci-ta/0.2/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# rpm-ostree-oci-ta task

RPM Ostree (Trusted Artifacts variant).

## Parameters
|name|description|default value|required|
|---|---|---|---|
|BUILDER_IMAGE|The location of the rpm-ostree builder image.|quay.io/redhat-user-workloads/project-sagano-tenant/ostree-builder/ostree-builder-fedora-38:d124414a81d17f31b1d734236f55272a241703d7|false|
|COMMIT_SHA|The image is built from this commit.|""|false|
|CONFIG_FILE|The relative path of the file used to configure the rpm-ostree tool found in source control. See https://github.com/coreos/rpm-ostree/blob/main/docs/container.md#adding-container-image-configuration|""|false|
|CONTEXT|Path to the directory to use as context.|.|false|
|HERMETIC|Determines if build will be executed without network access.|false|false|
|IMAGE|Reference of the image rpm-ostree will produce.||true|
|IMAGE_EXPIRES_AFTER|Delete image tag after specified time. Empty means to keep the image tag. Time values could be something like 1h, 2d, 3w for hours, days, and weeks, respectively.|""|false|
|IMAGE_FILE|The file to use to build the image||true|
|PLATFORM|The platform to build on||true|
|SOURCE_ARTIFACT|The Trusted Artifact URI pointing to the artifact with the application source code.||true|
|TLSVERIFY|Verify the TLS on the registry endpoint (for push/pull to a non-TLS registry)|true|false|

## Results
|name|description|
|---|---|
|IMAGE_DIGEST|Digest of the image just built|
|IMAGE_REF|Image reference of the built image|
|IMAGE_URL|Image repository and tag where the built image was pushed|
|SBOM_BLOB_URL|Reference, including digest to the SBOM blob|

11 changes: 11 additions & 0 deletions task/rpm-ostree-oci-ta/0.2/recipe.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
base: ../../rpm-ostree/0.2/rpm-ostree.yaml
add:
- use-source
preferStepTemplate: true
removeWorkspaces:
- source
replacements:
workspaces.source.path: /var/workdir/source
description: |-
RPM Ostree (Trusted Artifacts variant).
303 changes: 303 additions & 0 deletions task/rpm-ostree-oci-ta/0.2/rpm-ostree-oci-ta.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,303 @@
---
apiVersion: tekton.dev/v1
kind: Task
metadata:
name: rpm-ostree-oci-ta
annotations:
tekton.dev/pipelines.minVersion: 0.12.1
tekton.dev/tags: image-build, konflux
labels:
app.kubernetes.io/version: "0.1"
build.appstudio.redhat.com/build_type: rpm-ostree
build.appstudio.redhat.com/multi-platform-required: "true"
spec:
description: RPM Ostree (Trusted Artifacts variant).
params:
- name: BUILDER_IMAGE
description: The location of the rpm-ostree builder image.
type: string
default: quay.io/redhat-user-workloads/project-sagano-tenant/ostree-builder/ostree-builder-fedora-38:d124414a81d17f31b1d734236f55272a241703d7
- name: COMMIT_SHA
description: The image is built from this commit.
type: string
default: ""
- name: CONFIG_FILE
description: The relative path of the file used to configure the rpm-ostree
tool found in source control. See https://github.com/coreos/rpm-ostree/blob/main/docs/container.md#adding-container-image-configuration
type: string
default: ""
- name: CONTEXT
description: Path to the directory to use as context.
type: string
default: .
- name: HERMETIC
description: Determines if build will be executed without network access.
type: string
default: "false"
- name: IMAGE
description: Reference of the image rpm-ostree will produce.
type: string
- name: IMAGE_EXPIRES_AFTER
description: Delete image tag after specified time. Empty means to keep
the image tag. Time values could be something like 1h, 2d, 3w for
hours, days, and weeks, respectively.
type: string
default: ""
- name: IMAGE_FILE
description: The file to use to build the image
type: string
- name: PLATFORM
description: The platform to build on
type: string
- name: SOURCE_ARTIFACT
description: The Trusted Artifact URI pointing to the artifact with
the application source code.
type: string
- name: TLSVERIFY
description: Verify the TLS on the registry endpoint (for push/pull
to a non-TLS registry)
type: string
default: "true"
results:
- name: IMAGE_DIGEST
description: Digest of the image just built
- name: IMAGE_REF
description: Image reference of the built image
- name: IMAGE_URL
description: Image repository and tag where the built image was pushed
- name: SBOM_BLOB_URL
description: Reference, including digest to the SBOM blob
volumes:
- name: ssh
secret:
optional: false
secretName: multi-platform-ssh-$(context.taskRun.name)
- name: varlibcontainers
emptyDir: {}
- name: workdir
emptyDir: {}
stepTemplate:
env:
- name: BUILDER_IMAGE
value: $(params.BUILDER_IMAGE)
- name: CONFIG_FILE
value: $(params.CONFIG_FILE)
- name: CONTEXT
value: $(params.CONTEXT)
- name: HERMETIC
value: $(params.HERMETIC)
- name: IMAGE
value: $(params.IMAGE)
- name: IMAGE_EXPIRES_AFTER
value: $(params.IMAGE_EXPIRES_AFTER)
- name: IMAGE_FILE
value: $(params.IMAGE_FILE)
- name: TLSVERIFY
value: $(params.TLSVERIFY)
volumeMounts:
- mountPath: /var/workdir
name: workdir
steps:
- name: use-trusted-artifact
image: quay.io/redhat-appstudio/build-trusted-artifacts:latest@sha256:bc10298bff7805d8bc98211cd4534b9720f365f35ce0ef263dd65802de7ff036
args:
- use
- $(params.SOURCE_ARTIFACT)=/var/workdir/source
- name: build
image: quay.io/redhat-appstudio/multi-platform-runner:01c7670e81d5120347cf0ad13372742489985e5f@sha256:246adeaaba600e207131d63a7f706cffdcdc37d8f600c56187123ec62823ff44
workingDir: /var/workdir/source
volumeMounts:
- mountPath: /var/lib/containers
name: varlibcontainers
- mountPath: /ssh
name: ssh
readOnly: true
env:
- name: COMMIT_SHA
value: $(params.COMMIT_SHA)
script: |
#!/bin/bash
set -o verbose
set -eu
set -o pipefail
mkdir -p ~/.ssh
if [ -e "/ssh/error" ]; then
#no server could be provisioned
cat /ssh/error
exit 1
elif [ -e "/ssh/otp" ]; then
curl --cacert /ssh/otp-ca -XPOST -d @/ssh/otp "$(cat /ssh/otp-server)" >~/.ssh/id_rsa
echo "" >>~/.ssh/id_rsa
else
cp /ssh/id_rsa ~/.ssh
fi
chmod 0400 ~/.ssh/id_rsa
SSH_HOST="$(cat /ssh/host)"
export SSH_HOST
BUILD_DIR="$(cat /ssh/user-dir)"
export BUILD_DIR
SSH_ARGS="-o StrictHostKeyChecking=no"
export SSH_ARGS
mkdir -p scripts
echo "$BUILD_DIR"
# shellcheck disable=SC2086
ssh $SSH_ARGS "$SSH_HOST" mkdir -p "$BUILD_DIR/workspaces" "$BUILD_DIR/scripts" "$BUILD_DIR/tmp"

rsync -ra /var/workdir/source/ "$SSH_HOST:$BUILD_DIR/workspaces/source/"
cat >scripts/script-build.sh <<'REMOTESSHEOF'
#!/bin/sh
set -o verbose
set -eu
set -o pipefail
cd /var/workdir/source
if [ -z "$CONFIG_FILE" ] ; then
CONFIG_FILE_ARG=""
else
CONFIG_FILE_ARG=" --image-config=source/$CONFIG_FILE "
fi

prefetched_rpms_for_my_arch="./cachi2/output/deps/rpm/$(uname -m)"
if [ -d "$prefetched_rpms_for_my_arch" ]; then
# move all repo files out of the source repository to avoid conflicts with the cachi2.repo
mkdir /tmp/original-repos
find ./source -maxdepth 1 -name '*.repo' -exec mv {} /tmp/original-repos \;

# copy the platform-specific cachi2.repo into the source repository
cp "$prefetched_rpms_for_my_arch/repos.d/cachi2.repo" ./source

# set up cleanup handler
trap 'rm ./source/cachi2.repo; cp -r /tmp/original-repos/. ./source' EXIT

# link the cachi2 output dir to the expected location
# (the prefetch task expects the output to be at /cachi2/output during the build)
mkdir /cachi2
ln -s "$(realpath ./cachi2/output)" /cachi2/output
fi

rpm-ostree compose image --initialize --format oci $CONFIG_FILE_ARG "source/$IMAGE_FILE" rhtap-final-image
REMOTESSHEOF

if [ "$HERMETIC" = "true" ]; then
network_opt="--network=none"
else
network_opt=""
fi

chmod +x scripts/script-build.sh
rsync -ra scripts "$SSH_HOST:$BUILD_DIR"
rsync -ra "$HOME/.docker/" "$SSH_HOST:$BUILD_DIR/.docker/"
# shellcheck disable=SC2086
ssh $SSH_ARGS "$SSH_HOST" \
podman run \
$network_opt \
--mount type=bind,source="${BUILD_DIR}/tmp,target=/var/tmp,relabel=shared" \
--privileged \
-e CONTEXT="$CONTEXT" \
-e IMAGE_FILE="$IMAGE_FILE" \
-e CONFIG_FILE="$CONFIG_FILE" \
-e IMAGE="$IMAGE" \
-e IMAGE_EXPIRES_AFTER="$IMAGE_EXPIRES_AFTER" \
-e COMMIT_SHA="$COMMIT_SHA" \
--rm \
-v "$BUILD_DIR/workspaces/source:/var/workdir/source:Z" \
-v "${BUILD_DIR}/scripts":/script:Z \
-v "$BUILD_DIR/.docker/:/root/.docker:Z" \
--user=0 \
--entrypoint bash \
"$BUILDER_IMAGE" \
/script/script-build.sh
rsync -ra "$SSH_HOST:$BUILD_DIR/workspaces/source/" "/var/workdir/source/"
cp -r rhtap-final-image /var/lib/containers/rhtap-final-image
buildah pull oci:rhtap-final-image
buildah images
buildah tag localhost/rhtap-final-image "$IMAGE"
computeResources:
limits:
memory: 512Mi
requests:
cpu: 250m
memory: 128Mi
securityContext:
capabilities:
add:
- SETFCAP
- name: sbom-syft-generate
image: registry.access.redhat.com/rh-syft-tech-preview/syft-rhel9:1.4.1@sha256:34d7065427085a31dc4949bd283c001b91794d427e1e4cdf1b21ea4faf9fee3f
workingDir: /var/workdir/source/source
volumeMounts:
- mountPath: /var/lib/containers
name: varlibcontainers
script: |
syft oci-dir:/var/lib/containers/rhtap-final-image --output cyclonedx-json=/var/workdir/source/sbom-cyclonedx.json
computeResources:
limits:
memory: 6Gi
requests:
memory: 6Gi
- name: merge-cachi2-sbom
image: quay.io/redhat-appstudio/sbom-utility-scripts-image@sha256:53a3041dff341b7fd1765b9cc2c324625d19e804b2eaff10a6e6d9dcdbde3a91
workingDir: /var/workdir/source
script: |
cachi2_sbom=./cachi2/output/bom.json
if [ -f "$cachi2_sbom" ]; then
echo "Merging contents of $cachi2_sbom into sbom-cyclonedx.json"
python3 /scripts/merge_cachi2_sboms.py "$cachi2_sbom" sbom-cyclonedx.json >sbom-temp.json
mv sbom-temp.json sbom-cyclonedx.json
else
echo "Skipping step since no Cachi2 SBOM was produced"
fi
securityContext:
runAsUser: 0
- name: inject-sbom-and-push
image: quay.io/redhat-appstudio/multi-platform-runner:01c7670e81d5120347cf0ad13372742489985e5f
workingDir: /var/workdir/source
volumeMounts:
- mountPath: /var/lib/containers
name: varlibcontainers
script: |
#!/bin/bash
base_image_name=$(buildah inspect --format '{{ index .ImageAnnotations "org.opencontainers.image.base.name"}}' "${IMAGE}" | cut -f1 -d'@')
base_image_digest=$(buildah inspect --format '{{ index .ImageAnnotations "org.opencontainers.image.base.digest"}}' "${IMAGE}")
container=$(buildah from --pull-never "${IMAGE}")
buildah config -a org.opencontainers.image.base.name="${base_image_name}" -a org.opencontainers.image.base.digest="${base_image_digest}" "${container}"
buildah commit "${container}" "${IMAGE}"

status=-1
max_run=5
sleep_sec=10
for run in $(seq 1 $max_run); do
status=0
[ "$run" -gt 1 ] && sleep $sleep_sec
echo "Pushing sbom image to registry"
buildah push \
--tls-verify="${TLSVERIFY}" \
--digestfile "/var/workdir/source/image-digest" "${IMAGE}" \
"docker://${IMAGE}" && break || status=$?
done
if [ "$status" -ne 0 ]; then
echo "Failed to push sbom image to registry after ${max_run} tries"
exit 1
fi

<"/var/workdir/source"/image-digest tee "$(results.IMAGE_DIGEST.path)"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something in the shell formatter does this, it's a generated file and this is mostly esthetics so I don't think it is worth troubleshooting, but yell at me if you think it matters...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is shfmt the tool used?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

echo -n "$IMAGE" | tee "$(results.IMAGE_URL.path)"
{
echo -n "${IMAGE}@"
cat "/var/workdir/source/image-digest"
} >"$(results.IMAGE_REF.path)"

# Remove tag from IMAGE while allowing registry to contain a port number.
sbom_repo="${IMAGE%:*}"
sbom_digest="$(sha256sum sbom-cyclonedx.json | cut -d' ' -f1)"
# The SBOM_BLOB_URL is created by `cosign attach sbom`.
echo -n "${sbom_repo}@sha256:${sbom_digest}" | tee "$(results.SBOM_BLOB_URL.path)"
securityContext:
capabilities:
add:
- SETFCAP
runAsUser: 0
- name: upload-sbom
image: quay.io/konflux-ci/appstudio-utils:ab6b0b8e40e440158e7288c73aff1cf83a2cc8a9@sha256:24179f0efd06c65d16868c2d7eb82573cce8e43533de6cea14fec3b7446e0b14
workingDir: /var/workdir/source
script: |
cosign attach sbom --sbom sbom-cyclonedx.json --type cyclonedx "$(cat "$(results.IMAGE_REF.path)")"
6 changes: 6 additions & 0 deletions task/rpm-ostree-oci-ta/OWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@

#See the OWNERS docs: https://go.k8s.io/owners
approvers:
- cgwalters
reviewers:
- cgwalters
Loading
Loading