From 16130fbd00e269ac25375d70fd076759433c05a6 Mon Sep 17 00:00:00 2001 From: Senthilkumar Panneerselvam Date: Tue, 22 Oct 2024 13:00:24 +0200 Subject: [PATCH] Update readme --- README.md | 48 +++++++++++++++++++++++++++++++++--------------- 1 file changed, 33 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 416ca3f..4a675c8 100644 --- a/README.md +++ b/README.md @@ -1,34 +1,56 @@ DC dynamic ========== -This repository contains code to generate a docker container for running jobs. By default it runs the commands in `all.sh` when started. +This repository contains scripts to be run as cron jobs. `ACCESS_KEY` should be set for the container, defining the access key to use for Blobserver. -There are two cronjobs active on the cluster. No persistent storage is provided, so any generated files should be uploaded somewhere to be saved. +No persistent storage is provided, so any generated files should be uploaded somewhere to be saved. -### cron-daily +### cron run every 10 minutes -Runs the commands in the file `runner_daily.sh` on `0 4 * * * ` (04:00 every morning). +#### freya_runner_every10mins.sh + +Runs the scripts created by team Freya on `*/10 * * * *` (Every ten minutes) Any files put in `$CODE_PATH/output` will be uploaded to Blobserver using the dc-dynamic account. -### cron-weekly +### cron run hourly + +#### slack_runner_hourly.sh + +Runs the script `slack_feeds.py` on `40 * * * *` (Every hour at 40th minute) + +**slack_feeds.py** - Script to post message about new feeds (events/jobs) on SciLifeLab web to configured slack channel -Runs the commands in the file `runner_weekly.sh` on `0 4 * * 5 ` (04:00 every Friday morning). +### cron run daily + +#### freya_runner_daily.sh + +Runs the script created by team Freya on `0 7 * * *` (07:00 GMT). Any files put in `$CODE_PATH/output` will be uploaded to Blobserver using the dc-dynamic account. -### cron-every10minutes +#### slack_runner_daily.sh + +Runs the script `slack_figshare.py` on `0 6 * * *` (06:00 GMT) -Runs the commands in the file `runner_every10mins.sh` on `*/10 * * * * ` (Every ten minutes) +**slack_figshare.py** - Script to post message about new items in figshare to configured slack channel + +### cron run weekly + +#### freya_runner_weekly.sh + +Runs the script created by team Freya on `0 4 * * 5 ` (04:00 GMT every Friday morning). + +Any files put in `$CODE_PATH/output` will be uploaded to Blobserver using the dc-dynamic account. ## Adding jobs -Add new commands to the above files to have them running on schedule. If you need jobs to run on a different schedule, talk to @talavis. +Add new commands to the above files to have them running on schedule. If you need jobs to run on a different schedule, talk to Sys admins or team Freya. -Make sure that the `requirements.txt` file in dc-dynamic contains all modules required for your code **and** that any changes do not break the current code. +Make sure that the `requirements.txt` file in dc-dynamic contains all modules required for the native scripts (i.e. residing in dc-dynamic repo) **and** that any changes do not break the current code. ### Uploading files @@ -38,8 +60,4 @@ Th easiest way to upload files is to use curl: curl "https://blobserver.dc.scilifelab.se/blob/$filename" -H "x-accesskey: $ACCESS_KEY" --upload-file "$filename" ``` -Feel free to set new variables, but remember to tell @talavis if you need them to be set for the cronjobs on the cluster. - -## Eventual aim for this repository - -Files for running jobs should be held here. We should link to scripts elsewhere to avoid the duplication of e.g. visualisation scripts. +Feel free to set new variables, but remember to tell Sys admins or team Freya if you need them to be set for the cronjobs on the cluster.