Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement] Add a new data source to automate retrieving images from a image-list file of a GitHub release #543

Closed
PhanLe1010 opened this issue Dec 26, 2023 · 2 comments

Comments

@PhanLe1010
Copy link
Member

PhanLe1010 commented Dec 26, 2023

Current issue:

Longhorn would like to automate adding/mirroring images. However, the currently available list of data sources don't fit the use-case of Longhorn:

  1. github-releases data source: This one finds new GitHub release tag and adds the images defined in the images field in the config.json file. For example:
    {
      "vsphere-cpi": {
        "images": [
          "gcr.io/cloud-provider-vsphere/cpi/release/manager"
        ],
        "versionSource": "github-releases:kubernetes/cloud-provider-vsphere",
        "versionConstraint": ">1.21.0"
      }
    }
    This config.json file instructs the script to find GitHub release tags in the repo kubernetes/cloud-provider-vsphere. Then only add the gcr.io/cloud-provider-vsphere/cpi/release/manager image with the found tags to the images-list. This doesn't fit the use-case of Longhorn because the list of Longhorn images are not fixed. We added/removed images between the releases. Therefore, it would require manual works to modify the "images" fied of the config.json oftenly
  2. github-latest-release data source: this data source has same limitation as the github-releases. Additionaly, Longhorn maintain multiple minor releases so a smaller version (e.g., v1.4.5) might be released after the current latest version (e.g., v1.5.3). This data source will not sync and add the smaller version (e.g., v1.4.5)
  3. registry data source: Longhorn doesn't maintain a registry. Not applicable
  4. helm-latest, helm-oci, and helm-directory data sources. With these data sources, the workflow attempts to run helm template and extract the images from the workload (deployment/daemonset/pod) output of helm template. This approach doesn't work for Longhorn because not all Longhorn images appears in the output of Helm template (the images of Longhorn system managed components)

Proposal

Add a new data source to automate retrieving images from a image-list file of a GitHub release, github-releases-images-file. This will look up GitHub releases, excluding pre-releases, and find the list of images inside a specified file of the release. This can be used if your project maintains a list of images in a file, e.g., https://github.com/longhorn/longhorn/blob/master/deploy/longhorn-images.txt

An example of configuration for github-releases-images-file could be:

{
  "longhorn": {
    "versionSource": "github-releases-images-file:longhorn/longhorn",
    "imagesFilePath": "deploy/longhorn-images.txt",
    "versionConstraint": ">=1.4.0"
  }
}

With the new github-releases-images-file data source, the above config.json instruct the workflow to:

  1. Look up GitHub releases at the repo longhorn/longhorn, excluding pre-releases
  2. Only consider the releases which are >=1.4.0
  3. For each release, download the image list at deploy/longhorn-images.txt and add the newly found images to the images-list.
PhanLe1010 added a commit to PhanLe1010/image-mirror that referenced this issue Dec 26, 2023
…automate retrieving images from a image-list file of a GitHub release

rancher#543

Signed-off-by: Phan Le <phan.le@suse.com>
@PhanLe1010
Copy link
Member Author

PR created #544

PhanLe1010 added a commit to PhanLe1010/image-mirror that referenced this issue Jan 9, 2024
…tomate retrieving images from a image-list file of a GitHub release

rancher#543

Signed-off-by: Phan Le <phan.le@suse.com>
PhanLe1010 added a commit to PhanLe1010/image-mirror that referenced this issue Jan 9, 2024
…tomate retrieving images from a image-list file of a GitHub release

rancher#543

Signed-off-by: Phan Le <phan.le@suse.com>
brandond pushed a commit that referenced this issue Jan 10, 2024
…tomate retrieving images from a image-list file of a GitHub release

#543

Signed-off-by: Phan Le <phan.le@suse.com>
Copy link
Contributor

This repository uses an automated workflow to automatically label issues which have not had any activity (commit/comment/label) for 60 days. This helps us manage the community issues better. If the issue is still relevant, please add a comment to the issue so the workflow can remove the label and we know it is still valid. If it is no longer relevant (or possibly fixed in the latest release), the workflow will automatically close the issue in 14 days. Thank you for your contributions.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Mar 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant