Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set storage directory to make subdirectory. #428

Open
JoranAngevaare opened this issue Apr 18, 2021 · 0 comments
Open

Set storage directory to make subdirectory. #428

JoranAngevaare opened this issue Apr 18, 2021 · 0 comments

Comments

@JoranAngevaare
Copy link
Contributor

We should make the the storage directory make a sub directory.

For example:

# From
016389-raw_records-rfzvpzj4mf/
# To
016/016389-raw_records-rfzvpzj4mf/

If we allow generic characters for the run-numbers, each head e.g. 016 in this case, the main directory will have at most 26^3=17576 directories/datatype/lineage. From ~10000 onward one might expect FS slowness due to this.

We might very well make this an configurable option.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant