-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add default resource params and slurm destination #61
Conversation
tools.yml
Outdated
@@ -10,6 +12,8 @@ tools: | |||
rules: [] | |||
rank: | | |||
helpers.weighted_random_sampling(candidate_destinations) | |||
default: | |||
extends: tpvdb_default |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if default inherits tbvdb_default, what do this do that is different?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The idea was to preserve backward compatibility, but I think that this may end up just creating an infinite loop. We probably want this these changes to spread to all destinations anyway, so I'll change this back.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
This looks good. The way it is structured, would it be an opt-out rather than opt-in setup? |
5cd448c
to
c204c7e
Compare
c204c7e
to
5b58f29
Compare
tools.yml
Outdated
abstract: true | ||
tpvdb_slurm: | ||
context: | ||
time: '' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm wondering if these variable names are too generic. 'time' in particular could collide with python's time library
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a good point. I've renamed time for now, but perhaps we could just prefix everything with slurm_
instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the prefixing idea.
tools.yml
Outdated
context: | ||
time: '' | ||
partition: '' | ||
account: '' | ||
params: | ||
native_specification: --nodes=1 --ntasks={cores} --mem={round(mem*1024)} {'--time=' | ||
+ time if time else ''} {'--gres=gres:gpu:' + str(gpus) if gpus else ''} {'--partition=' | ||
+ partition if partition else ''} {'--account=' + account if account else | ||
''} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
context: | |
time: '' | |
partition: '' | |
account: '' | |
params: | |
native_specification: --nodes=1 --ntasks={cores} --mem={round(mem*1024)} {'--time=' | |
+ time if time else ''} {'--gres=gres:gpu:' + str(gpus) if gpus else ''} {'--partition=' | |
+ partition if partition else ''} {'--account=' + account if account else | |
''} | |
context: | |
walltime: '' | |
partition: '' | |
account: '' | |
params: | |
native_specification: --nodes=1 --ntasks={cores} --mem={round(mem*1024)} {'--time=' | |
+ walltime if walltime else ''} {'--gres=gres:gpu:' + str(gpus) if gpus else ''} {'--partition=' | |
+ partition if partition else ''} {'--account=' + account if account else | |
''} |
tools.yml
Outdated
abstract: true | ||
tpvdb_slurm: | ||
context: | ||
time: '' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a good point. I've renamed time for now, but perhaps we could just prefix everything with slurm_
instead?
tools.yml
Outdated
partition: '' | ||
account: '' | ||
params: | ||
native_specification: --nodes=1 --ntasks={cores} --mem={round(mem*1024)} {'--time=' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
given that this is multiline, it could be clearer to use one argument per line
--nodes=1
--ntasks={cores}
etc
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was because oftpv format
. I've tried an alternative formatting that at least preserves everything in one, very lengthy line. Setting the block chomping indicator to strip newlines also gets reformatted, so the only options appear to be either a single lengthy line or multiple lines broken up as above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fair enough. I hadn't thought about tpv format. This looks good as is
I'm not familiar with SLURM, to me it looks good. Thanks @nuwang |
closes: galaxyproject/total-perspective-vortex#114