0.3.7

• Tibanna unicorn is now independent of dcicutils.
• Unicorn not throwing error because it was set to pass it to the last step of pony is now fixed.

0.3.6

• Major changes
  • Use of dcicutils instead of Submit4dn
  • CWL v1 support

0.3.5

• postrunjson now patched to ffmeta
• rerun, rerun_many and kill_all added as invoke methods
• launch_utils added
• framework for metadata run_status update when error happens at 2nd/3rd step.
• an issue with secondary input file handling is fixed
• some issues with pseudo workflow run got fixed

0.3.4

• Now handling pseudo-workflows (when output_files are provided, tibanna creates a pseudo workflow_run object and links it to the output files, without actually running a workflow)

0.3.3

• removed SBG
• now takes experiments or source_experiments field from input file metadata and adds to the output processed file metadata
• now reports total space used for input/temp/output files to postrun.json and step function output.
• fixed an issue that the update_ffmeta_awsf step does not pass everything.

0.3.2

• A couple of changes have been made for nice handling of Pairsqc.
  • secondary input file handling
  • the last step of awsem (update_ffmeta_awsf) is updated to have integrated qc parser and updater to handle both fastqc and pairsqc

v0.3.1

• Tibanna branch (url) can be specified for awsf functions inside an input json file.
• Custom fields in CWL are now accepted.

v0.3

This release is nearly completely different from the previous release.

• AWS step functions are used as an upstream scheduler.
• The older awsf component is now working as a lambda function which generates an autonomous instance called awsem.
• Command-line launch uses invoke.
• Jupyter notebooks are used for connecting between steps and communicating with 4DN fourfront. (This functionality is not meant for general public)

v0.2

Version v0.2 has been tested.

Changes made:

• The whole process is now automated. The script create_json.py will create a json file, upload to s3, launch an instance that executes the workflow, sends logs to s3 and self-terminates. It uses the .tibanna.config file for storing relatively invariant parameters (e.g. reference bucket name, AMI, keyname, etc) and takes input files, parameters, etc as command-line options. It also adds the launched instance information to a job_list file to enable easier tracking.
• minor change in the json format. (the previous format may not work especially for a multi-layer cwl)

v0.1

v0.1 is tested on aws.
Changes made:

• Scripts/cwls are pulled from github instead of S3
• intermittent logging to S3 (not in 'real time', but still better than nothing)
• Large EBS (>2.5TB) can be mounted. EBS is now mounted to a nonroot directory so that large EBS volumes can be attached (2.5TB EBS can't be attached to root).
• Different EBS types are handled and especially io1 type with 5000 IOPS is set to default (very fast data transfer between S3 and EBS)
• Several directory handling issues related to cwl/docker (that came with the new nonroot EBS) were resolved
• run_workflow.(JOBID).sh is now automatically created by create_run_workflow.sh
• Minor format change in input json. (previous one doens't work anymore)