Skip to content

Commit

Permalink
Merge pull request #37 from ScilifelabDataCentre/update_readme
Browse files Browse the repository at this point in the history
Update readme
  • Loading branch information
misterabdullahAziz authored Oct 22, 2024
2 parents b453ab7 + 16130fb commit 266259c
Showing 1 changed file with 33 additions and 15 deletions.
48 changes: 33 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,56 @@
DC dynamic
==========

This repository contains code to generate a docker container for running jobs. By default it runs the commands in `all.sh` when started.
This repository contains scripts to be run as cron jobs.

`ACCESS_KEY` should be set for the container, defining the access key to use for Blobserver.

There are two cronjobs active on the cluster. No persistent storage is provided, so any generated files should be uploaded somewhere to be saved.
No persistent storage is provided, so any generated files should be uploaded somewhere to be saved.

### cron-daily
### cron run every 10 minutes

Runs the commands in the file `runner_daily.sh` on `0 4 * * * ` (04:00 every morning).
#### freya_runner_every10mins.sh

Runs the scripts created by team Freya on `*/10 * * * *` (Every ten minutes)

Any files put in `$CODE_PATH/output` will be uploaded to Blobserver using the dc-dynamic account.

### cron-weekly
### cron run hourly

#### slack_runner_hourly.sh

Runs the script `slack_feeds.py` on `40 * * * *` (Every hour at 40th minute)

**slack_feeds.py** - Script to post message about new feeds (events/jobs) on SciLifeLab web to configured slack channel

Runs the commands in the file `runner_weekly.sh` on `0 4 * * 5 ` (04:00 every Friday morning).
### cron run daily

#### freya_runner_daily.sh

Runs the script created by team Freya on `0 7 * * *` (07:00 GMT).

Any files put in `$CODE_PATH/output` will be uploaded to Blobserver using the dc-dynamic account.

### cron-every10minutes
#### slack_runner_daily.sh

Runs the script `slack_figshare.py` on `0 6 * * *` (06:00 GMT)

Runs the commands in the file `runner_every10mins.sh` on `*/10 * * * * ` (Every ten minutes)
**slack_figshare.py** - Script to post message about new items in figshare to configured slack channel

### cron run weekly

#### freya_runner_weekly.sh

Runs the script created by team Freya on `0 4 * * 5 ` (04:00 GMT every Friday morning).

Any files put in `$CODE_PATH/output` will be uploaded to Blobserver using the dc-dynamic account.


## Adding jobs

Add new commands to the above files to have them running on schedule. If you need jobs to run on a different schedule, talk to @talavis.
Add new commands to the above files to have them running on schedule. If you need jobs to run on a different schedule, talk to Sys admins or team Freya.

Make sure that the `requirements.txt` file in dc-dynamic contains all modules required for your code **and** that any changes do not break the current code.
Make sure that the `requirements.txt` file in dc-dynamic contains all modules required for the native scripts (i.e. residing in dc-dynamic repo) **and** that any changes do not break the current code.

### Uploading files

Expand All @@ -38,8 +60,4 @@ Th easiest way to upload files is to use curl:
curl "https://blobserver.dc.scilifelab.se/blob/$filename" -H "x-accesskey: $ACCESS_KEY" --upload-file "$filename"
```

Feel free to set new variables, but remember to tell @talavis if you need them to be set for the cronjobs on the cluster.

## Eventual aim for this repository

Files for running jobs should be held here. We should link to scripts elsewhere to avoid the duplication of e.g. visualisation scripts.
Feel free to set new variables, but remember to tell Sys admins or team Freya if you need them to be set for the cronjobs on the cluster.

0 comments on commit 266259c

Please sign in to comment.