Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reading an image collection from s3 with intake #3

Open
metazool opened this issue Jul 1, 2024 · 1 comment
Open

Reading an image collection from s3 with intake #3

metazool opened this issue Jul 1, 2024 · 1 comment
Labels
wontfix This will not be worked on

Comments

@metazool
Copy link
Collaborator

metazool commented Jul 1, 2024

All the scivision examples including the CEFAS plankton one seem to have image collections in a single zipfile.

In theory you can use a wildcard in a url_path but for me this throws an s3 error on the directory listing. I'm not sure whether this is a permissions issue or not actually a feature of intake.

At the moment, it's somewhat moot due to:

So for prototyping purposes intake is returning a CSV with file locations in s3, but we lose the benefit of the neatly packaged ImageSource - one to bear in mind when refactoring the image preprocessing and improving the metadata. Also worth requesting an object store from JASMIN for testing purposes, to experiment with the bucket policies.

@metazool metazool added the wontfix This will not be worked on label Sep 11, 2024
@metazool
Copy link
Collaborator Author

See also #25 and #29 - there's a good case for either dropping intake support or not building upon it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

1 participant