Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

somehow got application/vnd.oci.image.layer.v1.tar+gzip with zstd:chunked annotations #2053

Open
cgwalters opened this issue Aug 4, 2024 · 1 comment
Labels
area/zstd:chunked Issues relating to zstd:chunked bug

Comments

@cgwalters
Copy link
Contributor

I was messing around with zstd:chunked and somehow ended up with this:

skopeo inspect -n docker://quay.io/cgwalters/ostest:zstdchunked
{
    "Name": "quay.io/cgwalters/ostest",
...
    "LayersData": [
        {
            "MIMEType": "application/vnd.oci.image.layer.v1.tar+zstd",
            "Digest": "sha256:b8f741b50c9da370750578605c48ada1a67d01eea3bdae9ce09599c44dad7bfc",
            "Size": 3493473,
            "Annotations": {
                "io.github.containers.zstd-chunked.manifest-checksum": "sha256:01fb915db203d50de642297c259d4be864ee4cf67cd7ec46499d1fba31592820",
                "io.github.containers.zstd-chunked.manifest-position": "2429250:219805:2292554:1",
                "io.github.containers.zstd-chunked.tarsplit-checksum": "sha256:c976711edd7fc3b79ad01e6aceb4e2432f59b93fedbe7abea0e00d08e67b5c2b",
                "io.github.containers.zstd-chunked.tarsplit-position": "2649063:844338:13966956"
            }
        },
        {
            "MIMEType": "application/vnd.oci.image.layer.v1.tar+gzip",
            "Digest": "sha256:5a7e2a9026d1072b92565a640e04337d470fb0d8681563e461b1ecbeb18b548b",
            "Size": 401246016,
            "Annotations": {
                "io.github.containers.zstd-chunked.manifest-checksum": "sha256:3f91e5690651c9a09f9531443396cb3415f96130769192d99688d1102062d865",
                "io.github.containers.zstd-chunked.manifest-position": "400005112:761805:5527253:1",
                "io.github.containers.zstd-chunked.tarsplit-checksum": "sha256:59f55a3be0ef27a9d846773617795ae70279018ec8d2959d57d1456ab2ce3886",
                "io.github.containers.zstd-chunked.tarsplit-position": "400766925:934100:17583014"
            }
        },

There's a mix of gzip and zstd compressed layers; the gzip layers have the zstd-chunked annotations (which is nonsensical obviously).

As far as I can tell, the layers are compressed the way they claim to be (i.e. the gzip layers really are gzip, not zstd; so the bug here seems to be us choosing gzip for some reason (blob caching?))

This is 100% reproducible for me right now, but I haven't dug deep into why it's happening yet.

@cgwalters cgwalters added bug area/zstd:chunked Issues relating to zstd:chunked labels Aug 4, 2024
@mtrmac
Copy link
Collaborator

mtrmac commented Aug 10, 2024

This is almost certainly c/image; guessing, perhaps https://github.com/containers/image/blob/8c7c58c5aacd70fe8bc25da54f966a59baf175b0/copy/single.go#L895-L897 .

Anyway, a specific reproducer (which starts with removing the relevant blob-info-cache-v1.sqlite) would be useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/zstd:chunked Issues relating to zstd:chunked bug
Projects
None yet
Development

No branches or pull requests

2 participants