0.17.2

• IAM class now inheritable in stepfunction and core (no change in function)

0.17.1

• If EC2 still booting error persists for more than 10 minutes, the instance is terminated. This resolves the problem of some of these instances idly running even after the step function run declared failure. The change requires redeployment.

tibanna deploy_core -n check_task_awsem [-g <group>][-s <suffix>]

0.17.0

• CheckTaskAwsem now parses log and raises a more informative error message for certain cases at least, including:

  • No space left in device (either input download failure, docker pull failure or other space issue)
  • CWL missing input

• custom_errors field is now accepted in the input json args to let users define a custom, job-specific error messages based on log file. Tibanna will find pattern in the log file and report an error as the specified error_type. If pattern is multi-line, specify multiline: True (default False).

  • example:

    "custom_errors": [
        {
           "error_type": "Unmatching pairs in fastq"
           "pattern": "paired reads have different names: .+", 
           "multiline": false
        }
    ]
    

These changes require redeployment.

tibanna deploy_core -n check_task_awsem -g <usergroup> [-s <suffix>]
tibanna deploy_core -n run_task_awsem -g <usergroup> [-s <suffix>]

0.16.0

Now we have a tibanna cleanup function that removes all tibanna components. (also a part of core API)

tibanna cleanup -g <usergroup> [-s <suffix>] [--purge-history]
# --purge-history deletes all the records and history including logs, run jsons, post-run jsons, etc.
# (Do not use this option unless you're sure you want to delete all of them)

Example)

tibanna cleanup -g default_3647

Other minor changes

• The following two changes require redeployment.

  • Handling cases where EC2 instances have no Public IP (e.g. running on private VPC).
  • DynamoDB entries now have time stamp

  tibanna deploy_core -n run_task_awsem -g <group> [-s <suffix>]
  

0.15.6

• "security_group": <security_group_id> and/or "subnet": <subnet_id> can be added to config to specify security group and/or subnet.
• Redeployment is required for this change to take effect.

tibanna deploy_core -n run_task_awsem -g <group> [-s <sufifix>]

0.15.3

• step function error handling for lambda resource not found error
  • if a lambda is redeployed while a job is running, sometimes step function tries to access the lambda during the very short period of time after the lambda function is deleted and before an updated function is uploaded. In this rare case, step functions would throw a lambda resource not found error. From 0.15.3, this error is ignored and retried a few more times. In short, the currently running jobs are not sensitive to redeployment any more.
• Redeploy the whole unicorn for this to take effect.

tibanna deploy_unicorn -b buckets [-g groupname] [-s suffix] 

0.15.2

• A newly introduced bug in 0.15.1 in check_output and check_status core API fixed
• Now core API info(jobid) can be used to get information from dynamoDB for a given job ID (CLI not available yet). The following information is available.
  • exec_name
  • exec_arn
  • step function name
  • instance_id (only available after instance is launched)
  • log_bucket

0.15.1

• check_output and check_status API functions can now take job_id as an argument.

0.15.0

• With a previous version, if an instance is idle (CPU utilization < 1%) for an hour, Tibanna checks and kills it. This could happen rarely if an output file transfer is taking more than 1hr on a large machine with e.g. 72 cores, in which case the condition for idleness is satisfied but it should not be killed. From 0.15.0, Tibanna also checks EBS read metrics and if maximum 1kb is being read (e.g. for copying a file to s3) during the past hour, Tibanna does not kill the instance even though CPU utilization is <1%.

• Redeployment is required for this change to take effect.

tibanna deploy_core -n check_task_awsem [-g groupname] [-s suffix]

0.14.1

• Problem of not recognizing memory fixed
• Redeployment is required for this change to take effect.

tibanna deploy_core -n run_task_awsem -g <groupname> [-s <suffix>]