Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preliminary support for HTTP_AUTH. #84

Merged
merged 1 commit into from
Jul 4, 2024

Conversation

mattmoor
Copy link
Member

@mattmoor mattmoor commented Jul 3, 2024

I am still working through a few issues, but I believe the main problem at this point is that we don't support range requests.

@mattmoor mattmoor marked this pull request as draft July 3, 2024 01:34
apko/private/apk.bzl Outdated Show resolved Hide resolved
@mattmoor
Copy link
Member Author

mattmoor commented Jul 3, 2024

It looks like Alpine withdrew a package breaking one of the legs?

Error in download: java.io.IOException: Error downloading [https://dl-cdn.alpinelinux.org/alpine/v3.18/main/aarch64/libcrypto3-3.1.5-r0.apk] to /home/runner/.bazel/external/_main~apko~examples_multi_arch_and_repo_libcrypto3_aarch64_3.1.5-r0/https%3A%2F%2Fdl-cdn.alpinelinux.org%2Falpine%2Fv3.18%2Fmain/aarch64/libcrypto3-3.1.5-r0/06951e2d641fbcc40718707542c45a30ca1807f6.sig.tar.gz: GET returned 404 Not Found

@imjasonh
Copy link
Member

imjasonh commented Jul 3, 2024

It looks like Alpine withdrew a package breaking one of the legs?

Error in download: java.io.IOException: Error downloading [https://dl-cdn.alpinelinux.org/alpine/v3.18/main/aarch64/libcrypto3-3.1.5-r0.apk] to /home/runner/.bazel/external/_main~apko~examples_multi_arch_and_repo_libcrypto3_aarch64_3.1.5-r0/https%3A%2F%2Fdl-cdn.alpinelinux.org%2Falpine%2Fv3.18%2Fmain/aarch64/libcrypto3-3.1.5-r0/06951e2d641fbcc40718707542c45a30ca1807f6.sig.tar.gz: GET returned 404 Not Found

We should probably migrate tests to only pull from Wolfi, where we withdraw less often and have more control in general.

@mattmoor
Copy link
Member Author

mattmoor commented Jul 3, 2024

This fixes alpine for now by just regenerating the lock, but I agree we should switch to something where we have more control: #85

@mattmoor mattmoor force-pushed the support-http-auth branch 2 times, most recently from c11d57d to eb1a25e Compare July 4, 2024 00:34
I am still working through a few issues, but I believe the main problem at this point is that we don't support range requests.

Signed-off-by: Matt Moore <mattmoor@chainguard.dev>
@mattmoor mattmoor marked this pull request as ready for review July 4, 2024 00:43
@mattmoor mattmoor enabled auto-merge (squash) July 4, 2024 00:45
@mattmoor mattmoor merged commit 001025f into chainguard-dev:main Jul 4, 2024
10 checks passed
@mattmoor mattmoor deleted the support-http-auth branch July 4, 2024 02:41
@sfc-gh-mhazy
Copy link
Contributor

@mattmoor Thank you for the changes. We are quite excited about using this instead of building on top of base image!

I am wondering if we could use similar approach for fetching token within bazel rules as in rules_oci so that the rule don't depend on external env vars. I don't know much details about chainctl, but I am happy to implement the auth in bazel if you think this is feasible.

@mattmoor
Copy link
Member Author

@sfc-gh-mhazy I'm receptive to that. I added HTTP_AUTH mostly for symmetry with apko, melange and apk, but we'd love something better.

I bet @imjasonh and @jonjohnsonjr will have thoughts, but open to ideas for what we'd do here!

@imjasonh
Copy link
Member

apko and TF-apko have support for calling chainctl to provide auth for packages, which I think we should support in rules_apko too. They also have code support for doing Chainguard identity auth without chainctl based on env vars, if that's interesting to you.

@sfc-gh-mhazy
Copy link
Contributor

@imjasonh I think ideally the bazel rules would

  1. Not depend on chainctl being on host
    • If feasible, fetch the token as in rules_oci, without chainctl command at all. rules_oci seem to work for private chainguard images registries, not sure if this is a good sign here.
    • we could add chainctl toolchain (similar as apko toolchain)
  2. Don't fetch the token as part of build, because we want to support "offline" mode.
  3. We will need to think how to work with apko_lock which is not hermetic and serves for refreshing purposes.

In rules_apko, the apk fetch happens completely outside of apko, as part of apk.bzl. Perhaps we should fetch token there ("natively" with rctx.download or with chainctl)

Maybe lets move the discussion to some doc?

@imjasonh
Copy link
Member

Thanks, this is really great context.

I think the fix will be for apk.bzl to fetch the token (using chainctl or CHAINGUARD_IDENTITY env var, either way), and populate the cache for offline image builds.

Then, when building the image in offline mode, we should (we might already) not attempt to get a token until we're actually about to use it to fetch an apk -- which we won't do if offline mode. I think this might already work this way, or if not, it could fairly easily. This would be a performance benefit even in online-mode, since a token wouldn't be requested if there are no APKs to fetch if they're already in the cache.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants