relecov-tools v1.2.0

Release Summary for relecov-tools v1.2.0 - 2024-10-11

Modules:

• Wrapper Module: Introduced to launch download, read-lab-metadata, and validate processes sequentially.
• Pipeline Manager: Renamed and updated from launch-pipeline, with enhanced functionality when using tools via CLI.

Enhancements:

• Integrity Checks: Added gzip file integrity check post-download; moved cleaning process to the workflow's end.
• Reconnection Decorator: Introduced in sftp_client.py to handle lost connections.
• Hospital Additions: Added Hospital Universitari Doctor Josep Trueta and Hospital Universitario Araba to laboratory_address.json.
• Metadata Handling:
  • samples_data.json is no longer mandatory as input for read-lab-metadata.
  • Supports alternative column names for distinct headers in read-lab-metadata.
  • Generates new samples_data.json if not provided.
  • Handles missing sample IDs and improved logging.
• Cleaning Process: More accurate by skipping only sequencing files instead of entire folders.
• Logs and Summaries: Single log summaries created for each folder during download; enhanced log handling for download, read-lab-metadata, and wrapper.
• Excel and Messages: Added Excel resize, bracket removal in messages, and improved exception handling in log_summary.py.

Fixes:

• City Name Correction: Corrected wrong city name in laboratory_address.json.
• Layout Detection: Fixed single-paired layout detection issues due to capital letters.
• Error Handling: Improved error handling in merge_logs() and create_logs_excel() methods.
• Empty Rows: Handled multiple empty rows in metadata Excel files.

Changes:

• Configuration Updates:
  • Refactored bioinfo_lab_heading to alt_header_equivalences in configuration.json.
  • Added missing schema fields related to bioinformatics results.
• Metadata Template: Updated and moved to relecov_tools/assets.
• Python Lint: Triggered only when PR includes Python files.
• Concurrency and Testing: Moved concurrency to the whole workflow and updated test datasets.
• Download Skipping: Folders with "invalid_samples" in their name are now skipped during download.
• Warnings and Labels: Some warnings in read-lab-metadata now include labels, and trailing spaces removed.
• Pipeline Data Splitting: Pipeline manager now splits data based on enrichment_panel and version, organizing into separate folders.

Removed:

• Duplicate Tests: Removed redundant tests in test_sftp-handle.
• Deprecated Auto-Release: Deleted non-functional auto-release in pypi_publish.
• Reconnection Decorator: Removed initial sleep time for the reconnection decorator in sftp_client.py, with increased sleep time from the second attempt.

relecov-tools v1.1.0

[v1.1.0] - 2024-09-16

Credits

Code contributions to the hotfix:

• Pablo Mata
• Sara Monzón

Modules

• New logs-to-excel function to create an excel file given a list of log-summary.json files #300

Added enhancements

• Included a way to extract pango-designation version in read-bioinfo-metadata #299
• Now log_summary.py also creates an excel file with the process logs #300
• Read-bioinfo-metadata splits files and data by batch of samples #306
• Included a sleep time in test_sftp-handle to avoid concurrency check failure #308

Fixes

• Fixes in launch_pipeline including creation of samples_id.txt and joined validated json #303
• Fixed failing module_tests.yml workflow due to deprecated upload-artifact version #308

Changed

• Changed pypi_publish action to publish on every release, no need to push tags #308

Removed

• Removed only_samples argument in log_summary.py as it was not used in any module. #300

relecov-tools v1.0.0

This release is the first fully functional version of the relecov-tools package. This means that the workflow for data management can be executed end-to-end with no restrictions from now on.

First dev version

Full phage+ schema, before reducing fields.