-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dataexamples tidy #1752
base: master
Are you sure you want to change the base?
dataexamples tidy #1752
Conversation
1681b09
to
479b83e
Compare
976dd26
to
bc3bef2
Compare
bc3bef2
to
017a4cf
Compare
''' | ||
URL = 'https://zenodo.org/records/4912435/files/small.zip' | ||
FILE_SIZE = '227 MB' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@hrobarts I think you're missing a @classmethod def get(...)
here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not obvious which file you would want to get from the sandstone dataset. I was thinking of just having the download function then the user can load whichever one they need. That's how I would use it in the how-to. But maybe that's a bad idea. What do you think @paskino ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could add an extra argument to the get
function here to select the file?
loader = ZEISSDataReader(file_name=filepath) | ||
return loader.read() | ||
except FileNotFoundError as exc: | ||
raise ValueError(f"Specify a different data_dir or download data with `{cls.__name__}({data_dir}).download_data()`") from exc |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
btw @hrobarts why not this instead? It will prompt if not already downloaded...
try: | |
loader = ZEISSDataReader(file_name=filepath) | |
return loader.read() | |
except FileNotFoundError as exc: | |
raise ValueError(f"Specify a different data_dir or download data with `{cls.__name__}({data_dir}).download_data()`") from exc | |
self.download_data() | |
loader = ZEISSDataReader(file_name=filepath) | |
return loader.read() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We had some discussion about wanting to keep the download separate from get so the user has to actively request the download. But I agree as we have the prompt before downloading this would be neater...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also could have def get(..., missing='prompt' / 'download' / 'error')
to control (default) behaviour
Co-authored-by: Casper da Costa-Luis <casper.dcl@physics.org> Signed-off-by: Hannah Robarts <77114597+hrobarts@users.noreply.github.com>
return res == "y" | ||
|
||
def download_data(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we sometimes want to just download the data without getting it e.g. in the how-to for the readers, the purpose is to show the loading step. In that case I think we need to keep data_dir as an argument and maybe keep this as a class method??
FYI |
a910556
to
f8b513b
Compare
Changes
OOP tidy of #1712
abc.ABC
->BaseTestData
->TestData,NexusTestData,RemoteTestData
**kwargs
(Use of **kwargs in methods and functions signature is confusing #1115)_download_and_extract_from_url
Testing you performed
Related issues/links
Checklist
I have updated the relevant documentationCHANGELOG.md has been updated with any functionality changeContribution Notes
Please read and adhere to the developer guide and local patterns and conventions.