diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md deleted file mode 100644 index 752d800..0000000 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -name: Bug report -about: Create a report to help us improve TomoBEAR -title: "[BUG] give a descriptive name here" -labels: '' -assignees: '' - ---- - -**Describe the bug** - -A clear and concise description of what the bug is - -**Project JSON file** - -```JSON -Place the contents of the JSON file here -``` - -**Error message** - -Place the error message here - -**To Reproduce** -Steps to reproduce the behavior: -1. Step -2. Step -3. See error - -**Expected behavior** -A clear and concise description of what you expected to happen. - -**Screenshots** -If applicable, add screenshots to help explain your problem. - -**Desktop (please complete the following information):** - - OS: [e.g. iOS] - - OS Version [e.g. 22] - - MotionCor2 Version - - Gctf Version - - Dynamo Version - - -**Additional context** -Add any other context about the problem here. diff --git a/.github/ISSUE_TEMPLATE/issue_report.yml b/.github/ISSUE_TEMPLATE/issue_report.yml new file mode 100644 index 0000000..0f60d27 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/issue_report.yml @@ -0,0 +1,101 @@ +name: Issue Report +description: Report a bug, ask for support or make feature request +title: "Put issue title here" +labels: ["pending"] +body: + - type: dropdown + id: issue-type + attributes: + label: Issue type + description: What type of issue would you like to report? + multiple: false + options: + - Bug + - Build/Install + - Performance + - Support + - Feature Request + - Documentation Feature Request + - Documentation Bug + - Others + validations: + required: true + - type: dropdown + id: tomobear-source + attributes: + label: TomoBEAR source + description: TomoBEAR is used from + multiple: true + options: + - source code (via interactive MATLAB session) + - standalone binary (using MATLAB libraries) + validations: + required: true + - type: input + id: tomobear-version + attributes: + label: TomoBEAR version + description: Which TomoBEAR version do you use? + placeholder: v0.4.0 + validations: + required: true + - type: markdown + attributes: + value: | + If you cloned specific branch or commit, please execute "git log --pretty="%H - %cd" -n 1" in your cloned TomoBEAR source code folder and put its output in the field above + + - type: dropdown + id: computing-system + attributes: + label: Computing system + description: Which kind of computing system(s) do you use? + multiple: true + options: + - Standalone GPU workstation + - HPC cluster (interactive computing node) + - HPC cluster (via scheduler submission) + validations: + required: true + + - type: textarea + id: issue-description + attributes: + label: Issue description + description: Provide a clear description of what happend and what did you expect to happen. + placeholder: Tell us what you see! + validations: + required: true + - type: textarea + id: configs + attributes: + label: Input configuration (JSON) file + description: Please copy and paste your input configuration (JSON) file. Use + value: "```JSON { ... } ```" + validations: + required: true + + - type: textarea + id: additional-info + attributes: + label: Additional info + description: Add any other context about the problem here (actions to reproduce, log/error messages, example data, additional configurations, screenshots, links/references). + + - type: input + id: gpu-info + attributes: + label: GPU model and memory + description: if the problem might be related to the GPU parallelization + placeholder: NVIDIA Ampere A40-48GB + - type: input + id: cuda-info + attributes: + label: CUDA version(s) + description: if the problem might be related to the GPU parallelization + placeholder: CUDA-11.5 + + - type: textarea + id: software-info + attributes: + label: Software setup + description: Give here list of software and their versions (and other relevant info) which may be related to the issue (e.g. MATLAB, IMOD, Dynamo, MotionCor2, AreTomo, Gctf, IsoNet, etc.) + placeholder: MATLAB-2021a, IMOD-4.11.24; Gctf-v1.18; etc. diff --git a/AUTHORS.md b/AUTHORS.md index 1039c56..b6f8b27 100644 --- a/AUTHORS.md +++ b/AUTHORS.md @@ -1,16 +1,13 @@ ## TomoBEAR authors -Max Delbruck Center for Molecular Medicine, Berlin, Germany () +Copyright (c) 2022 Max Delbruck Center for Molecular Medicine, Berlin, Germany () * Mikhail Kudryashev - project leader * Artsemi Yushkevich - developer * Vasilii Mikirtumov - application engineer +Copyright (c) 2021 Max Planck Institute of Biophysics, Frankfurt (Hesse), Germany () -Max Planck Institute of Biophysics, Frankfurt (Hesse), Germany () - +* Mikhail Kudryashev - project leader * Nikita Balyschew - core version developer - -## External modified software authors - -TomoBEAR also contains modified pieces of MATLAB source code of the Dynamo package developed by D. Castaño-Díez et al.: https://www.dynamo-em.org. +* Vasilii Mikirtumov - application engineer diff --git a/CHANGELOG.md b/CHANGELOG.md index 5fb16fa..21ff132 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,30 @@ All notable changes will be documented in this file. +## v0.4.0 - 2023-08-07 + +### :rocket: New Features +* ```crYOLO``` - module wrapping [crYOLO](https://cryolo.readthedocs.io/en/stable/index.html) - DL tool, based on **You Only Look Once (YOLO)** object detection system for predicting particle coordinates (basically picking them) in cryo-electron tomograms + +### :arrow_up: Improvements +* ```IsoNet```: added possibility to post-process IsoNet-corrected tomograms using NAD filtering +* ```DynamoTemplateMatching```: enabled usage of the tomograms from the custom user-provided path +* ```AreTomo```: + * enabled direct usage of AreTomo alignment parameters from previously executed AreTomo step + * enabled parameters: tilt axis offset (pre-tilt) and temporary reconstruction thickness (VolZ) + +### :bug: Important fixes +* ```DynamoTemplateMatching```: fixed bug preventing GPU-enabled speedup + +### :exclamation: Important updates +* **changed license** of the core code base from [`AGPLv3`](https://www.gnu.org/licenses/agpl-3.0.en.html) to [`GPLv3`](https://www.gnu.org/licenses/gpl-3.0.en.html) - changed license to a bit more permissive version in order to allow software maintainers and other TomoBEAR contributors more flexibility in terms of TomoBEAR modifications, needed to provide a TomoBEAR operation over local networks (like institutional intranets and/or shared filesystems); +* **updated documentation**: + * recorded a [set of short 8-12 min video-tutorials](https://www.youtube.com/watch?v=2uizkE616tE&list=PLdMU06ILLrYKjI-Z0qezcNheEtpVulMao&pp=iAQB) on TomoBEAR installation and 80S ribosome (EMPIAR-10064) tutorial; + * added [contribution guidelines and tips](https://github.com/KudryashevLab/TomoBEAR/blob/main/CONTRIBUTING.md) + * extended ["Usage"](https://github.com/KudryashevLab/TomoBEAR/wiki/Usage) and reviewed ["Installation and Setup"](https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup) sections; + * added ["External Software"](https://github.com/KudryashevLab/TomoBEAR/wiki/External-Software) section for dependencies installation/citation description. + +--- ## v0.3.0 - 2023-05-22 ### :rocket: New Features diff --git a/CITATION.cff b/CITATION.cff index 324167a..0f55cc6 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -20,6 +20,6 @@ authors: given-names: Misha orcid: https://orcid.org/0000-0003-3550-6274 title: "Streamlined Structure Determination by Cryo-Electron Tomography and Subtomogram Averaging using TomoBEAR" -version: 0.3.0 +version: 0.4.0 doi: https://doi.org/10.1101/2023.01.10.523437 -date-released: 2023-05-22 +date-released: 2023-08-07 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..3dd596d --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,144 @@ +# Contributing guidelines + +## Foreword + +*Dear cryo-ET enthusiast,* +
we are happy that you are considering contribution to the `TomoBEAR` workflow, which is an [**open-source**](https://opensource.guide/) academic software. We do believe that community contribution and co-development efforts help to collect and bring in practice excellent ideas, overcome issues and initial design limitations thankfully to the broadness and diversity of the community experience and expertise as well as generally accelerate development in order to faster achieve software stabilization. That influences software life cycle outcomes and software lifetime expectancy, which in the end defines such important parameters for academic software as: +* re-usability +* utilization +* availability +* reproducibility +* transparency + +Those principles comply with [**FAIR4RS Principles**](https://www.nature.com/articles/s41597-022-01710-x), which we are trying to follow in the `TomoBEAR` project too. We do hope that the open-source state of this project along with opened communication channels enable a possibility to engage cryo-ET community into collaborative development. +
*Sincerely yours,* +
*TomoBEAR team* + +## Making your contribution + +To our contributors we suggest the broadly used **fork-and-pull** model which consists of the following steps: +1. Fork this project to your repository. +2. Make local changes and push them to your fork. +3. Prepare pull request back to our repository along with description of the made changes. +4. Give us time to review and decide on the future of your pull request. + +## General guidelines and principles + +Whenever contributing to `TomoBEAR`, please, consider the following: +* **Divide and conquer.** Do not contribute unconnected things together! It makes much harder for us to review and accept your contribution then. +* **Test locally and globally.** Along with isolated testing of the module/function which you change/contribute to, please, try to predict and test effect of your changes on the whole pipeline or at least on the "neighboring" modules, which are supposed to be executed right before/after the module you are contributing to. We do not provide unit-tests or other kind of tests to run yet, but we ask at least to run your changes on the data from our [Ribosome tutorial](https://github.com/KudryashevLab/TomoBEAR/wiki/Tutorials) or any other **EMPIAR** data set (e.g. one of those we [used in our preprint](https://www.biorxiv.org/content/10.1101/2023.01.10.523437v1)) or, at the best, on one of your real data sets. +* **Include license note.** If you add a new code file, include license note ([see example in Module.m](https://github.com/KudryashevLab/TomoBEAR/blob/develop/modules/Module.m)) at the top for [**GPLv3**](https://www.gnu.org/licenses/gpl-3.0.en.html) or any other **compatible with GPLv3** license. +* **Use proposed contribution model.** Use **fork-and-pull** model to bring changes for our review, please. This is the safest contribution mechanism for both contributors and reviewers. + +## TomoBEAR design + +If you are willing to contribute the bug-fix, a new feature/module or have an idea of how to re-design some part of the pipeline, there we provide some minimal developer/contributor notes regarding the `TomoBEAR` intermediate user-level and developer-level design which might be important to know. + +### User-level design + +Please, consider that good understanding of the user-level design of `TomoBEAR` is the entry-level requirement for reading these notes and contributing to `TomoBEAR`. For the details on basic user-level design you may refer to the corresponding sections of the [TomoBEAR wiki](https://github.com/KudryashevLab/TomoBEAR/wiki/Home) such as [Installation and Setup](https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup), [Tutorials](https://github.com/KudryashevLab/TomoBEAR/wiki/Tutorials) and [Usage](https://github.com/KudryashevLab/TomoBEAR/wiki/Usage). Here we provide some additional information for intermediate user-level design of the `TomoBEAR`: + +* **Configuration files.** There are two types of configuration files, from which all the metadata about pipeline and for main (experimental and processed) data is assembled and used during execution. Those are: + * `defaults.json` which contains all the global parameters and parameters for all the modules as well as their default values. + * `input.json` or any other filename of user's input configuration file for the particular project, which holds user's pipeline structure, metadata for user's primary project pipeline, execution and processing parameters values. + Data in the `input.json` (user's project file) has higher priority than data in the `defaults.json`. That means that if parameter field is absent in `input.json`, it will be taken from `defaults.json`, otherwise - from `input.json`. + +* **Execution modes.** Here are several execution modes which define the TomoBEAR workflow behavior, namely: + * `local` - (*stable*) data processing mode for offline (static) input data for sequential/parallel(CPU/GPU) processing locally or on the interactive cluster node; + * `cleanup` - (*feature in test*) not a data processing mode for cleaning up some metadata and intermediate results to save up space + * `local_live` - (*experimental feature*) data processing mode for online (live, on-the-fly) input data for sequential/parallel(CPU/GPU) processing locally or on the interactive cluster node; + * `slurm` - (*refactoring is planned*) data processing mode for offline (static) input data for parallel(CPU/GPU) processing by distributing tasks over cluster nodes using SLURM scheduler system. + +* **Pipeline execution checkpoint files.** There are execution checkpoint files, which `TomoBEAR` puts after each processing step in the corresponding folder: + * `SUCCESS`\\`FAILURE` which indicate the outcome of the whole processing step or operation applied to the particular tilt serie or tomogram (for the modules which process tilt series or tomograms independently of each other like `CreateStacks`, `MotionCor2`, `BatchRunTomo`, etc.). + > **SUCCESS files contents** + >
The file `SUCCESS` contains bit (0/1) vector of the number of tomograms length with 0's or 1's meaning failure or success of processing for the corresponding tilt serie or tomogram so that successful ones will be used on the further processing steps. This might be useful to know when you need workaround if something is not working. + + * `TIME` which contains the time in seconds that it took TomoBEAR to execute the certain step or process the particular tilt serie or tomogram. + +### Developer-level design + +**Project and code directories structure.** + +![Project/code structure](https://github.com/KudryashevLab/TomoBEAR/blob/main/images/TomoBEAR_file_structures.png) + +`TomoBEAR` *project folder* is structured as presented on the image above with the following elements: +* `input.json` - the input configuration file which users should create themselves; +* `output/` - the main output folder which contains all the kinds of data (input, intermediate, output) and metadata (for data and pipeline) structured in the following way: + * `XX_StepName_1/` - those are processing steps folders, containing in general step-related `data_files.*` and `data_folders/` (for example, individual tomogram/tilt serie folders `tomogram_*` for the modules like `CreateStacks`, `BinStacks`, `Reconstruct`), as well as step-related pipeline metadata files, like various JSON files (`input.json`, `output.json`, etc.), log file of the step execution `SteName.log`, pipeline checkpoint file `SUCCESS` or `FAILURE` (depending on processing outcome) and execution time log file `TIME`; + * `pipeline.log` - general log file, tracking global pipeline execution messages (info/warnings/errors); + * `data_folders/` and `metadata_folders/` - folders where the most important (meta)data is moved or linked to. +* `scratch/` - special folder which contains just folder tree of the `output/` folder reflecting workflow structure and serving as a reference during the pipeline execution. + +`TomoBEAR` *source code* is structured as presented on the image above and consists of the following *three main parts*: +* **environment setup** scripts/functions (`run_matlab.sh`, `startup.m`, `environment/`) - those are for configuring environmental and MATLAB Path variables +* **processing modules** (`modules/`) - those are interface modules for TomoBEAR steps, wrapping external software and/or our developments functionality +* **pipeline modules** (`pipeline/`) - those are pipeline execution modules for different computational environments + +as well as +* **modified external routines** (`dynamo/`) - special folder to merge main `Dynamo` source code with our re-implemented `Dynamo` routines during `TomoBEAR` configuration to virtually produce updated functional version of `Dynamo`; +* **auxilary functionality and files** (`utilities/`, `configuration/`, `configurations/` and `json/`) - variety of scripts/functions as well as templates and sets of defaults, used elsewhere in the briefly described above main parts; + +and, finally +* **documentation** (`wiki/`, `images/`, `README.md`, `CHANGELOG.md`, `CONTRIBUTING.md`, `AUTHORS.md`, `CITATION.cff`, `LICENSE.md`, `LICENSES/`) - users-oriented files describing how to configure, install and use `TomoBEAR` as well as developers-oriented files containing general information about `TomoBEAR` as an open-source project like authors and contributors list, licensing and citation information, list of changes by releases, contribution guidelines, etc. + +Whenever you open interactive MATLAB session in TomoBEAR folder, the first file to start is `startup.m` which is MATLAB-specific way of setting up environment. Afterwards, when you use `runTomoBEAR` command to initialize or execute TomoBEAR workflow, this runs `pipeline/runPipeline.m` function, which basically performs a set of additional setups, heads to the selected *computational environment*, starts it in the chosen *execution mode* and runs it via the requested *execution method*. The details on the available *computational environments* you may find in the section **Pipeline modules** below, *execution modes* - is in the section **User-level design** above, and *execution methods* is described in the section **Additional developer-level design elements** below. + +### Environment setup + +**Environment setup** scripts/functions include: +* `run_matlab.sh` - a bash script for TomoBEAR configuration to be executable in interactive MATLAB session, which sets up some system environment variables and opens MATLAB session; +* `startup.m` - is the script for MATLAB-specific way of setting up MATLAB environment, which is auto-executed every time as you open MATLAB from TomoBEAR source code folder; +* contents of `environment/` - several additional scripts to e.g. enable recognition of external software commands and conda environment paths. + +### Processing modules + +**Processing modules** include contents of the `modules/` directory. Those are interface modules for TomoBEAR steps, wrapping external software and/or our developments functionality. All the processing modules are inherited from the only parent abstract module called **Module** (`modules/Module.m`), implementing the core of the processing module routines such as definition and initialization of module properties (e.g. computing environment information, global/local/temporal configurations, input/output/log paths), metadata creation, collection (e.g. execution time measurement), writing (log files), and some metadata cleanup functionality. + +All the modules are quite different, but their contents depend on the own interface of the external programs/frameworks being wrapped. Thus, *processing modules* could be classified into groups +* **by domain knowledge** according to the cryo-ET/StA processing stage as the following: + * Pipeline behavior control modules + * Cryo-ET data processing modules (functionality from raw dose-fractionated movies up to and including tomographic reconstructions) + * Particles picking-associated modules (functionality of particles picking and extraction) + * Subtomogram Averaging modules (functionality of single-reference and multi-reference particles alignment and classification) +* **by interface type of the functionality** being wrapped as the following: + * metadata/files/execution management (`MetaData`, `SortFiles` and `StopPipeline`) + * standalone command line software (e.g. modules wrapping MotionCor2, CTFFIND4, GCTF, AreTomo, etc.) + * frameworks/libraries/packages (e.g. Dynamo/SUSAN/IMOD-oriented modules) + * original algorithms and data processing code pieces (e.g. `GridEdgeEraser`, `TemplateMatchingPostProcessing`, etc.) + * Neural network-based Python tool sets, containing pre-processing, training, prediction and post-processing procedures (e.g. `IsoNet`, `crYOLO`, etc.) + +Despite the plenty of possible ways to create a new module, depending on the original algorithm or external tool structure and execution features, we have prepared two mini-templates along with comments, which are located by the following relative paths: +* `modules/GeneralModuleTemplate` - a general-purpose processing module template; +* `modules/DLToolModuleTemplate` - a (Python-based) deep learning tool-oriented processing module template. + +as well as +* `configurations/defaults_template.json` - a configuration file, containing templates of sections for the two module templates above. + +We hope those mini-templates can help our future contributors to better understand structure of a typical `TomoBEAR` module in order to overcome initial contribution barrier. + +### Pipeline modules + +**Pipeline modules** include contents of the `pipeline/` directory. Those are pipeline execution modules for different *computational environments* defining the way how TomoBEAR runs/distributes processing modules execution over the available computational resources in the certain *computational environment*. All the pipeline modules are inherited from the abstract module **Pipeline** (`pipeline/Pipeline.m`). + +The available computational environments are described in the table below. + +| Title | Pipeline module | Target system | Execution mode | State of development | +| :------------- | :--- | :---- | :--- | :--- | +| **Local** | `pipeline/LocalPipeline.m` | a local workstation or a single (interactive) HPC cluster node | `local`/`cleanup` | stable | +| **Local live** | `pipeline/LocalLivePipeline.m` | a local workstation for live data processing | `local_live` | experimental | +| **SLURM** | `pipeline/SlurmPipeline.m` | HPC cluster using SLURM scheduler | `slurm`| requires refactoring | +| **Grid Engine** | `pipeline/GridEnginePipeline.m` | HPC cluster using SGE scheduler | N/A | in early development | + +### Additional developer-level design elements + +* **Execution methods.** The *execution methods* are mostly definitions of the parallelisation schemes and a certain *execution method* can be used by setting up the corresponding `"execution_method"` parameter value in the certain section of the configuration file corresponding to the target module. In a bit of details about each of them: + * `once` - a one-time execution of the module functional (e.g. used by metadata-collecting modules like `MetaData`, by data importing/exporting modules like `EMDTemplateGeneration`). However, be aware that this *execution method* is also used to execute modules with internal (in-module) parallelisation (e.g. when wrapped algorithm uses its own parallelisation scheme like modules `GenerateParticles`, `crYOLO` and `IsoNet` do). + * `sequential` - a sequential multi-time independent execution of the module functional on the different data chunks like tilt series or tomograms, when parallelization is not desired. As well, this method is useful and the only possibility for debugging modules which by default use parallel *execution methods* like `parallel` or `in_order`. + * `parallel` - a simultaneous multi-time independent execution of the module functional on the different data chunks like tilt series or tomograms, serves for functional **parallelized on CPU's** (e.g. modules `"SortFiles"`, `"DynamoCleanStacks"`, `"BinStacks"`); + * `in_order` - a simultaneous multi-time independent execution of the module functional on the different data chunks like tilt series or tomograms, serves for functional **parallelized on GPU's** (e.g. modules `"MotionCor2"`, `"AreTomo"`, `"Reconstruct"`, `"DynamoTemplateMatching"`); + * `control` - the special *execution method* reserved only for operation of the `"StopPipeline"` control module. + +* **Configuration variables.** There are two main configuration fields which you need to know about when writing a module: + * `obj.configuration` - this is a structure field which holds global (from `"general"` section) and local (from the corresponding module section) parameters and their values initialized with the higher priority to the value set in module-specific configuration section unless the latest is not set and the corresponding value from `"general"` section or value defined by the lowest-priority `configurations/defaults.json` is used; + * `obj.dynamic_configuration` - this is a "dynamic" structure field which is aimed at storing temporal or local parameters values for the currently executed module (processing step); the variables of the obj.configuration above at the next processing step are updated in accordance with this structure field `obj.dynamic_configuration`. diff --git a/LICENSE.md b/LICENSE.md index b475c3a..2ed2025 100644 --- a/LICENSE.md +++ b/LICENSE.md @@ -1,11 +1,9 @@ # License -Copyright © 2021-2023 Max Delbruck Center for Molecular Medicine (MDC) -
-Copyright © 2021 Max Planck Institute of Biophysics (MPIBP) +The list of **TomoBEAR Authors** as copyright holders of this work you may find in the **AUTHORS.md** file by the following location: https://github.com/KudryashevLab/TomoBEAR/blob/main/AUTHORS.md This work is licensed under multiple licenses: -- The source code (MATLAB, Python, Bash, etc.) and the accompanying material are licensed under [AGPL3](LICENSES/AGPL-3.0-or-later.txt). +- The source code (MATLAB, Python, Bash, etc.) and the accompanying material are licensed under [GPL3](LICENSES/GPL-3.0-or-later.txt). - The modified MATLAB source code of the Dynamo package (```dynamo/```) is licensed under [GPL2](LICENSES/GPL-2.0-or-later.txt). - The documentation (```wiki/```), TomoBEAR logo and accompanying images (```images/```) as well as configuration files (```configurations/```) are licensed under [CC-BY-4.0](LICENSES/CC-BY-4.0.txt). - Insignificant files are licensed under [CC0-1.0](LICENSES/CC0-1.0.txt). diff --git a/LICENSES/AGPL-3.0-or-later.txt b/LICENSES/AGPL-3.0-or-later.txt deleted file mode 100644 index 0c97efd..0000000 --- a/LICENSES/AGPL-3.0-or-later.txt +++ /dev/null @@ -1,235 +0,0 @@ -GNU AFFERO GENERAL PUBLIC LICENSE -Version 3, 19 November 2007 - -Copyright (C) 2007 Free Software Foundation, Inc. - -Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. - - Preamble - -The GNU Affero General Public License is a free, copyleft license for software and other kinds of works, specifically designed to ensure cooperation with the community in the case of network server software. - -The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, our General Public Licenses are intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. - -When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. - -Developers that use our General Public Licenses protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License which gives you legal permission to copy, distribute and/or modify the software. - -A secondary benefit of defending all users' freedom is that improvements made in alternate versions of the program, if they receive widespread use, become available for other developers to incorporate. Many developers of free software are heartened and encouraged by the resulting cooperation. However, in the case of software used on network servers, this result may fail to come about. The GNU General Public License permits making a modified version and letting the public access it on a server without ever releasing its source code to the public. - -The GNU Affero General Public License is designed specifically to ensure that, in such cases, the modified source code becomes available to the community. It requires the operator of a network server to provide the source code of the modified version running there to the users of that server. Therefore, public use of a modified version, on a publicly accessible server, gives the public access to the source code of the modified version. - -An older license, called the Affero General Public License and published by Affero, was designed to accomplish similar goals. This is a different license, not a version of the Affero GPL, but Affero has released a new version of the Affero GPL which permits relicensing under this license. - -The precise terms and conditions for copying, distribution and modification follow. - - TERMS AND CONDITIONS - -0. Definitions. - -"This License" refers to version 3 of the GNU Affero General Public License. - -"Copyright" also means copyright-like laws that apply to other kinds of works, such as semiconductor masks. - -"The Program" refers to any copyrightable work licensed under this License. Each licensee is addressed as "you". "Licensees" and "recipients" may be individuals or organizations. - -To "modify" a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a "modified version" of the earlier work or a work "based on" the earlier work. - -A "covered work" means either the unmodified Program or a work based on the Program. - -To "propagate" a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well. - -To "convey" a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying. - -An interactive user interface displays "Appropriate Legal Notices" to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. - -1. Source Code. -The "source code" for a work means the preferred form of the work for making modifications to it. "Object code" means any non-source form of a work. - -A "Standard Interface" means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language. - -The "System Libraries" of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A "Major Component", in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. - -The "Corresponding Source" for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those -subprograms and other parts of the work. - -The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source. - -The Corresponding Source for a work in source code form is that same work. - -2. Basic Permissions. -All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law. - -You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. - -Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary. - -3. Protecting Users' Legal Rights From Anti-Circumvention Law. -No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures. - -When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures. - -4. Conveying Verbatim Copies. -You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program. - -You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee. - -5. Conveying Modified Source Versions. -You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions: - - a) The work must carry prominent notices stating that you modified it, and giving a relevant date. - - b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to "keep intact all notices". - - c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it. - - d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so. - -A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an "aggregate" if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. - -6. Conveying Non-Source Forms. -You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways: - - a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange. - - b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. - - c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b. - - d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. - - e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d. - -A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. - -A "User Product" is either (1) a "consumer product", which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, "normally used" refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product. - -"Installation Information" for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made. - -If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM). - -The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network. - -Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. - -7. Additional Terms. -"Additional permissions" are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. - -When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. - -Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: - - a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or - - b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or - - c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or - - d) Limiting the use for publicity purposes of names of licensors or authors of the material; or - - e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or - - f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. - -All other non-permissive additional terms are considered "further restrictions" within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. - -If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms. - -Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way. - -8. Termination. - -You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). - -However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. - -Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice. - -Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10. - -9. Acceptance Not Required for Having Copies. - -You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so. - -10. Automatic Licensing of Downstream Recipients. - -Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. - -An "entity transaction" is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts. - -You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it. - -11. Patents. - -A "contributor" is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's "contributor version". - -A contributor's "essential patent claims" are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, "control" includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. - -Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version. - -In the following three paragraphs, a "patent license" is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To "grant" such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party. - -If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent -license to downstream recipients. "Knowingly relying" means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid. - -If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it. - -A patent license is "discriminatory" if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. - -Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law. - -12. No Surrender of Others' Freedom. - -If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may -not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program. - -13. Remote Network Interaction; Use with the GNU General Public License. - -Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software. This Corresponding Source shall include the Corresponding Source for any work covered by version 3 of the GNU General Public License that is incorporated pursuant to the following paragraph. - -Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the work with which it is combined will remain governed by version 3 of the GNU General Public License. - -14. Revised Versions of this License. - -The Free Software Foundation may publish revised and/or new versions of the GNU Affero General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. - -Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU Affero General Public License "or any later version" applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU Affero General Public License, you may choose any version ever published by the Free Software Foundation. - -If the Program specifies that a proxy can decide which future versions of the GNU Affero General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program. - -Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version. - -15. Disclaimer of Warranty. - -THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. - -16. Limitation of Liability. - -IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. - -17. Interpretation of Sections 15 and 16. - -If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. - -END OF TERMS AND CONDITIONS - - How to Apply These Terms to Your New Programs - -If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. - -To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. - - - Copyright (C) - - This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. - - This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. - - You should have received a copy of the GNU Affero General Public License along with this program. If not, see . - -Also add information on how to contact you by electronic and paper mail. - -If your software can interact with users remotely through a computer network, you should also make sure that it provides a way for users to get its source. For example, if your program is a web application, its interface could display a "Source" link that leads users to an archive of the code. There are many ways you could offer source, and different solutions will be better for different programs; see section 13 for the specific requirements. - -You should also get your employer (if you work as a programmer) or school, if any, to sign a "copyright disclaimer" for the program, if necessary. For more information on this, and how to apply and follow the GNU AGPL, see . diff --git a/LICENSES/GPL-3.0-or-later.txt b/LICENSES/GPL-3.0-or-later.txt new file mode 100644 index 0000000..d41c0bd --- /dev/null +++ b/LICENSES/GPL-3.0-or-later.txt @@ -0,0 +1,232 @@ +GNU GENERAL PUBLIC LICENSE +Version 3, 29 June 2007 + +Copyright © 2007 Free Software Foundation, Inc. + +Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. + +Preamble + +The GNU General Public License is a free, copyleft license for software and other kinds of works. + +The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. We, the Free Software Foundation, use the GNU General Public License for most of our software; it applies also to any other work released this way by its authors. You can apply it to your programs, too. + +When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things. + +To protect your rights, we need to prevent others from denying you these rights or asking you to surrender the rights. Therefore, you have certain responsibilities if you distribute copies of the software, or if you modify it: responsibilities to respect the freedom of others. + +For example, if you distribute copies of such a program, whether gratis or for a fee, you must pass on to the recipients the same freedoms that you received. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. + +Developers that use the GNU GPL protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License giving you legal permission to copy, distribute and/or modify it. + +For the developers' and authors' protection, the GPL clearly explains that there is no warranty for this free software. For both users' and authors' sake, the GPL requires that modified versions be marked as changed, so that their problems will not be attributed erroneously to authors of previous versions. + +Some devices are designed to deny users access to install or run modified versions of the software inside them, although the manufacturer can do so. This is fundamentally incompatible with the aim of protecting users' freedom to change the software. The systematic pattern of such abuse occurs in the area of products for individuals to use, which is precisely where it is most unacceptable. Therefore, we have designed this version of the GPL to prohibit the practice for those products. If such problems arise substantially in other domains, we stand ready to extend this provision to those domains in future versions of the GPL, as needed to protect the freedom of users. + +Finally, every program is threatened constantly by software patents. States should not allow patents to restrict development and use of software on general-purpose computers, but in those that do, we wish to avoid the special danger that patents applied to a free program could make it effectively proprietary. To prevent this, the GPL assures that patents cannot be used to render the program non-free. + +The precise terms and conditions for copying, distribution and modification follow. + +TERMS AND CONDITIONS + +0. Definitions. + +“This License” refers to version 3 of the GNU General Public License. + +“Copyright” also means copyright-like laws that apply to other kinds of works, such as semiconductor masks. + +“The Program” refers to any copyrightable work licensed under this License. Each licensee is addressed as “you”. “Licensees” and “recipients” may be individuals or organizations. + +To “modify” a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a “modified version” of the earlier work or a work “based on” the earlier work. + +A “covered work” means either the unmodified Program or a work based on the Program. + +To “propagate” a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well. + +To “convey” a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying. + +An interactive user interface displays “Appropriate Legal Notices” to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion. + +1. Source Code. +The “source code” for a work means the preferred form of the work for making modifications to it. “Object code” means any non-source form of a work. + +A “Standard Interface” means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language. + +The “System Libraries” of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A “Major Component”, in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it. + +The “Corresponding Source” for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work. + +The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source. + +The Corresponding Source for a work in source code form is that same work. + +2. Basic Permissions. +All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law. + +You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you. + +Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary. + +3. Protecting Users' Legal Rights From Anti-Circumvention Law. +No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures. + +When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures. + +4. Conveying Verbatim Copies. +You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program. + +You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee. + +5. Conveying Modified Source Versions. +You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to “keep intact all notices”. + + c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so. + +A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an “aggregate” if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate. + +6. Conveying Non-Source Forms. +You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways: + + a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b. + + d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d. + +A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work. + +A “User Product” is either (1) a “consumer product”, which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, “normally used” refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product. + +“Installation Information” for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made. + +If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM). + +The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network. + +Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying. + +7. Additional Terms. +“Additional permissions” are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions. + +When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission. + +Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or authors of the material; or + + e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors. + +All other non-permissive additional terms are considered “further restrictions” within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying. + +If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms. + +Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way. + +8. Termination. +You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11). + +However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation. + +Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice. + +Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10. + +9. Acceptance Not Required for Having Copies. +You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so. + +10. Automatic Licensing of Downstream Recipients. +Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License. + +An “entity transaction” is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts. + +You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it. + +11. Patents. +A “contributor” is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's “contributor version”. + +A contributor's “essential patent claims” are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, “control” includes the right to grant patent sublicenses in a manner consistent with the requirements of this License. + +Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version. + +In the following three paragraphs, a “patent license” is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To “grant” such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party. + +If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. “Knowingly relying” means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid. + +If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it. + +A patent license is “discriminatory” if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007. + +Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law. + +12. No Surrender of Others' Freedom. +If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program. + +13. Use with the GNU Affero General Public License. +Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU Affero General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the special requirements of the GNU Affero General Public License, section 13, concerning interaction through a network will apply to the combination as such. + +14. Revised Versions of this License. +The Free Software Foundation may publish revised and/or new versions of the GNU General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. + +Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU General Public License “or any later version” applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU General Public License, you may choose any version ever published by the Free Software Foundation. + +If the Program specifies that a proxy can decide which future versions of the GNU General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program. + +Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version. + +15. Disclaimer of Warranty. +THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + +16. Limitation of Liability. +IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. + +17. Interpretation of Sections 15 and 16. +If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee. + +END OF TERMS AND CONDITIONS + +How to Apply These Terms to Your New Programs + +If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. + +To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the “copyright” line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. + + This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. + + You should have received a copy of the GNU General Public License along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + +If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, your program's commands might be different; for a GUI interface, you would use an “about box”. + +You should also get your employer (if you work as a programmer) or school, if any, to sign a “copyright disclaimer” for the program, if necessary. For more information on this, and how to apply and follow the GNU GPL, see . + +The GNU General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. But first, please read . diff --git a/README.md b/README.md index fd79004..b1eb471 100644 --- a/README.md +++ b/README.md @@ -25,10 +25,10 @@ Implementation details and benchmarks you can find in our preprint: ### Video-tutorials We have prepared a range of short (8-12 min) video-tutorials explaining setup, usage and example output of the ```TomoBEAR``` to help you get started with ```TomoBEAR``` based on the [ribosome tutorial](https://github.com/KudryashevLab/TomoBEAR/wiki/Tutorials): -* [Video 1](https://drive.google.com/file/d/1Nhltdo7DbWdzPhAbnkzq7lCkDghxJrku/view?usp=share_link): how to get the latest ```TomoBEAR``` version and configure ```TomoBEAR``` and its dependencies; -* [Video 2](https://drive.google.com/file/d/12h8Vs4eUeJDJAb5S6nF2BuOX7lfvmmYo/view?usp=share_link): description of the project configuration file and the pipeline execution; -* [Video 3](https://drive.google.com/file/d/16nNdu89XUZglO_PiRKmGkK85o_nWqOB8/view?usp=share_link): additional configuration file parameters description, ```TomoBEAR```-```IMOD```-```TomoBEAR``` loop for checking tilt series alignment results and fiducials refinement (if needed); -* [Video 4](https://drive.google.com/file/d/1t3ol3KlHqBGtnZp1bnLHi_Anurd2VPcx/view?usp=share_link): checking on further intermediate results (alignment, CTF-correction, reconstruction, template matching). +* [Video 1](https://youtu.be/2uizkE616tE): how to get the latest ```TomoBEAR``` version and configure ```TomoBEAR``` and its dependencies; +* [Video 2](https://youtu.be/N93tfAXp990): description of the project configuration file and the pipeline execution; +* [Video 3](https://youtu.be/qbkRtMJp0eI): additional configuration file parameters description, ```TomoBEAR```-```IMOD```-```TomoBEAR``` loop for checking tilt series alignment results and fiducials refinement (if needed); +* [Video 4](https://youtu.be/BP2T_Y7BiDo): checking on further intermediate results (alignment, CTF-correction, reconstruction, template matching). ### Pipeline structure @@ -60,13 +60,13 @@ In case of any questions, issues or suggestions you may interact with us by one * write an e-mail to [Misha Kudryashev](mailto:misha.kudryashev@gmail.com) or [Artsemi Yushkevich](mailto:Artsemi.Yushkevich@mdc-berlin.de); * start a discussion in [Discussions](https://github.com/KudryashevLab/TomoBEAR/discussions); -If you wish to contribute, please, fork this repository and make a pull request back with your changes and a short description. +If you wish to contribute, please, fork this repository and make a pull request back with your changes and a short description. For further details on contribution plase read our [contribution guidelines](CONTRIBUTING.md). ## Citation If you use `TomoBEAR` or its parts in your research, please **cite both** `TomoBEAR` and **all external software packages** which you have used under `TomoBEAR`. -The `TomoBEAR` modules dependencies on third-party software are listed on the page [Modules](https://github.com/KudryashevLab/TomoBEAR/wiki/Modules.md) and the list of the corresponding references to cite is located on the page [Additional Software Citation](https://github.com/KudryashevLab/TomoBEAR/wiki/Additional-Software-Citation.md). +The `TomoBEAR` modules dependencies on third-party software are listed on the page [Modules](https://github.com/KudryashevLab/TomoBEAR/wiki/Modules.md) and the list of the corresponding references to cite is located on the page [External Software](https://github.com/KudryashevLab/TomoBEAR/wiki/External-Software.md). ## Acknowledgements @@ -76,6 +76,15 @@ We are grateful to the following organizations: As well we are grateful to the [structural biology scientific research group of Werner Kühlbrandt](https://www.biophys.mpg.de/2207989/werner_kuehlbrandt) at the [MPIBP (Max Planck Institute of Biophysics)](https://www.biophys.mpg.de) and the [MPIBP](https://www.biophys.mpg.de) in Frankfurt (Hesse), Germany for support. +The authors thank as well the following people: +* Dr. Daniel Castano-Diez, Dr. Kendra Leigh and Dr. Christoph Diebolder and Dr. Wolfgang Lugmayr for useful discussions; +* Uljana Kravchenko, Xiaofeng Chu, Giulia Glorani for testing the developmental versions and providing feedback, +* Ricardo Sanchez for producing MATLAB version of the [SUSAN framework](https://github.com/rkms86/SUSAN) in order to be compatible with TomoBEAR; +* Juan Castillo from the Max Planck Institute for Biophysics for the IT support at the Max Planck for Biophysics, +* the high-performance computing team at the MDC for supporting our operation at the Max-Cluster. + +We would like to acknowledge as well that TomoBEAR contains modified pieces of MATLAB source code of the Dynamo package developed by Dr. Daniel Castaño-Díez et al.: https://www.dynamo-em.org. + ## Contacts * Prof. Dr. Misha Kudryashev[^1][^2] ([e-mail](mailto:misha.kudryashev@gmail.com?subject=[GitHub]%20TomoBEAR)) - `TomoBEAR` project leader, Principal Investigator; diff --git a/configuration/ConfigurationParser.m b/configuration/ConfigurationParser.m index a376c37..490aecd 100644 --- a/configuration/ConfigurationParser.m +++ b/configuration/ConfigurationParser.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef ConfigurationParser < handle properties (Access = private) file_path string; diff --git a/configurations/defaults.json b/configurations/defaults.json index b5c260f..5a9716d 100755 --- a/configurations/defaults.json +++ b/configurations/defaults.json @@ -50,6 +50,7 @@ "tilting_step": 3, "gold_bead_size_in_nm": 10, "rotation_tilt_axis": 85, + "pretilt_tilt_axis": 0, "pid_wait_time": 5, "raw_files_folder": "raw_files", "fid_files_folder": "fiducial_models", @@ -285,12 +286,12 @@ "throw": "0", "trunc": "0", "sum_range": [], - "fm_ref": "", + "fm_ref": -1, "skip": false, "citation": "", "citation_link": "", "doi": "", - "eer_sampling": 2, + "eer_upsampling": 2, "fm_int_file": "", "eer_total_number_of_fractions": 576, "eer_fraction_grouping": 16, @@ -517,6 +518,7 @@ }, "DynamoTemplateMatching": { "execution_method": "in_order", + "use_custom_tomograms_path": "", "use_ctf_corrected_tomograms": true, "show_table": false, "show_cross_correlations": false, @@ -897,10 +899,8 @@ "reconstruction": false, "weighted_back_projection": true, "tilt_axis_refine_flag": 1, - "correct_tilt_axis_offset": 0, - "apply_given_tilt_axis_offset": false, - "tilt_axis_offset": 0, - "align_height_ratio": 0.75, + "correct_pretilt_flag": 1, + "align_z": -1, "apply_dose_weighting": false, "sart": "20 5", "roi": "0 0", @@ -908,6 +908,7 @@ "patch": "0 0", "flip_volume": 1, "flip_intensity": 0, + "use_previous_alignment": false, "citation": "" }, "RelionAlignmentProject": { @@ -1017,6 +1018,62 @@ "crop_size": -1, "batch_size": -1, "normalize_percentile": -1 + }, + "postprocess": { + "input_dir": "corrected_tomos", + "output_dir": "postproc_tomos", + "nad_filter_output_iterations_list": [3], + "nad_filter_number_of_iterations": -1, + "nad_filter_sigma_for_smoothing": -1, + "nad_filter_threshold_for_gradients": -1 + } + } + }, + "crYOLO": { + "execution_method": "once", + "cryolo_env": "", + "tomograms_to_use": [], + "steps_to_execute": {}, + "steps_to_execute_defaults": { + "config": { + "cryolo_command": "cryolo_gui.py config", + "target_boxsize": 220, + "config_json_filepath": "config_cryolo.json", + "filter": "LOWPASS", + "low_pass_cutoff": 0.1, + "janni_model_path": "", + "train_mode": false, + "train_tomograms_folder": "train_tomograms", + "train_tomograms_path": "", + "tomograms_binning": -1, + "train_annot_folder": "train_annot", + "input_size": -1 + }, + "train": { + "cryolo_command": "cryolo_train.py", + "config_json_filepath": "config_cryolo.json", + "num_cpu": -1, + "early": 15, + "warmup": 5 + }, + "predict": { + "cryolo_command": "cryolo_predict.py", + "config_json_filepath": "config_cryolo.json", + "trained_model_filepath": "cryolo_model.h5", + "threshold": 0.1, + "test_tomograms_folder": "test_tomograms", + "test_tomograms_path": "", + "tomograms_binning": -1, + "predict_annot_folder": "predict_annot", + "num_cpu": -1, + "tracing_search_range": -1, + "tracing_memory": -1, + "tracing_min_length": -1 + }, + "export_annotations": { + "raw_prtcl_coords_dir": "predict_annot/COORDS", + "per_table_particle_count": -1, + "total_particle_count": -1 } } } diff --git a/configurations/defaults_template.json b/configurations/defaults_template.json new file mode 100755 index 0000000..4e71e46 --- /dev/null +++ b/configurations/defaults_template.json @@ -0,0 +1,41 @@ +{ + "general": { + ... + "new_global_param_number": default_value, + "new_global_param_string": "default_value", + "new_global_params_list": [default_value1,...,default_valueN], + "new_global_params_section": { + ... // put here global params and their defaults + } + ... + }, + "GeneralModuleTemplate": { + "execution_method": "once", + "new_module_param_number": default_value, + "new_module_param_string": "default_value", + "new_module_params_list": [default_value1,...,default_valueN], + "new_module_params_section": { + ... // put here module params and their defaults + } + }, + "DLToolModuleTemplate": { + "execution_method": "once", + "dltool_env": "", + "repository_path": "", + ... // put here module params and their defaults + "tomograms_to_use": [X, X, X], + "steps_to_execute": {}, + "steps_to_execute_defaults": { + "step_name1": { + "tomograms_binning": X, + "ncpu": X, + "output_dir": "module_subdirectory_name", + ... // put here other params and defaults for step #1 + }, + ... + "step_nameN": { + ... // put here params and defaults for step #N + } + } + } +} \ No newline at end of file diff --git a/configurations/live_template.json b/configurations/live_template.json new file mode 100644 index 0000000..86e110d --- /dev/null +++ b/configurations/live_template.json @@ -0,0 +1,43 @@ +{ + "general": { + "project_name": "your_project_name", + "project_description": "your project name description", + "data_path": "/path/to/live/data/folder/prefix*.tif", + "processing_path": "/path/to/processing/folder", + "expected_symmetrie": "Cx", + "gold_bead_size_in_nm": xx, + "rotation_tilt_axis": xx, + "aligned_stack_binning": 8, + "pre_aligned_stack_binning": 8, + "reconstruction_thickness": xxxx, + "as_boxes": false, + "minimum_files": 41, + "ft_bin": 2, + "listening_time_threshold_in_minutes": 10 + }, + "MetaData": { + }, + "SortFiles": { + }, + "MotionCor2": { + "method": "SumOnly", + "execution_method": "sequential" + }, + "CreateStacks": { + }, + "DynamoTiltSeriesAlignment": { + "use_newstack_for_binning": true + }, + "DynamoCleanStacks": { + }, + "BatchRunTomo": { + "skip_steps": [4, 7, 9], + "ending_step": 13 + }, + "BinStacks": { + }, + "Reconstruct": { + "generate_exact_filtered_tomograms": true, + "exact_filter_size": xxxx + } +} \ No newline at end of file diff --git a/configurations/pfib_ribosome_empiar_11306.json b/configurations/pfib_ribosome_empiar_11306.json new file mode 100644 index 0000000..f9bc8d2 --- /dev/null +++ b/configurations/pfib_ribosome_empiar_11306.json @@ -0,0 +1,94 @@ +{ + "general": { + "project_name": "EMPIAR-11306", + "project_description": "Ribosome from FIB-milled data of EMPIAR-11306", + "data_path": "/path/to/raw/data/*.eer", + "processing_path": "/path/for/processing/", + "expected_symmetrie": "C1", + "aligned_stack_binning": 2, + "apix": 1.85, + "gpu": [x,x,x,x,x], + "binnings": [4, 8, 16], + "rotation_tilt_axis": 80, + "template_matching_binning": 8, + "as_boxes": false + }, + "MetaData": { + }, + "SortFiles": { + }, + "MotionCor2": { + "gain": "/path/to/reference/gain/20220406_175020_EER_GainReference.gain", + "eer_sampling": 2, + "eer_total_number_of_fractions": 343, + "eer_fraction_grouping": 34, + "eer_exposure_per_fraction": 0.5, + "ft_bin": 2 + }, + "CreateStacks": { + }, + "BatchRunTomo": { + "ending_step": 6, + "skip_steps": [5], + "directives": { + "comparam.xcorr.tiltxcorr.FilterRadius2": 0.07, + "comparam.xcorr.tiltxcorr.FilterSigma1": 0.00, + "comparam.xcorr.tiltxcorr.FilterSigma2": 0.02, + "comparam.xcorr.tiltxcorr.CumulativeCorrelation": 1, + "comparam.xcorr_pt.tiltxcorr.FilterRadius2": 0.07, + "comparam.xcorr_pt.tiltxcorr.FilterSigma1": 0.00, + "comparam.xcorr_pt.tiltxcorr.FilterSigma2": 0.02, + "comparam.xcorr_pt.tiltxcorr.IterateCorrelations": 1, + "comparam.xcorr_pt.tiltxcorr.SizeOfPatchesXandY": "500 500", + "comparam.xcorr_pt.tiltxcorr.OverlapOfPatchesXandY": "0.33 0.33", + "comparam.xcorr_pt.tiltxcorr.BordersInXandY": "102 102", + "runtime.Fiducials.any.trackingMethod": 1, + "runtime.AlignedStack.any.eraseGold": 0, + "comparam.align.tiltalign.RotOption": -1, + "comparam.align.tiltalign.MagOption": 0, + "comparam.align.tiltalign.TiltOption": 0, + "comparam.align.tiltalign.ProjectionStretch": 0 + } + }, + "BatchRunTomo": { + "starting_step": 8, + "ending_step": 8 + }, + "GCTFCtfphaseflipCTFCorrection": { + }, + "BatchRunTomo": { + "starting_step": 11, + "skip_steps": [12], + "ending_step": 13 + }, + "BinStacks": { + }, + "Reconstruct": { + "reconstruction_thickness": 3000, + "generate_exact_filtered_tomograms": true, + "exact_filter_size": 2500 + }, + "DynamoImportTomograms": { + }, + "TemplateGenerationFromFile": { + "template_path": "/path/to/template.mrc", + "mask_path": "/path/to/mask.mrc", + "template_apix": xx.x, + "use_smoothed_mask": false, + "use_bandpassed_template": false, + "use_ellipsoid": false + }, + "StopPipeline": { + }, + "DynamoTemplateMatching": { + "cone_range": 360, + "cone_sampling": 10, + "in_plane_range": 360, + "in_plane_sampling": 10, + "size_of_chunk": [512, 512, 375] + }, + "TemplateMatchingPostProcessing": { + "cc_std": 2.5, + "crop_particles": false + } +} \ No newline at end of file diff --git a/configurations/ribosome_empiar_10064_dynamo.json b/configurations/ribosome_empiar_10064_dynamo.json index 56cfd4c..985ab65 100644 --- a/configurations/ribosome_empiar_10064_dynamo.json +++ b/configurations/ribosome_empiar_10064_dynamo.json @@ -49,6 +49,8 @@ "flip_handedness": true }, "DynamoTemplateMatching": { + "sampling": 15, + "size_of_chunk": [463, 463, 175] }, "TemplateMatchingPostProcessing": { "cc_std": 2.5 diff --git a/dynamo/matlab/src/+dpktomo/+match/+aux/@Tile/compute.m b/dynamo/matlab/src/+dpktomo/+match/+aux/@Tile/compute.m index cf0346b..acd6a57 100755 --- a/dynamo/matlab/src/+dpktomo/+match/+aux/@Tile/compute.m +++ b/dynamo/matlab/src/+dpktomo/+match/+aux/@Tile/compute.m @@ -41,8 +41,9 @@ function compute(obj,varargin) Nmask = operationalData.NMask; % -chunkNormalized = dynamo_normalize_roi(chunk); - +gpuChunk = gpuArray(chunk); +chunkNormalized = dynamo_normalize_roi(gpuChunk); +chunkNormalized = gpuArray(chunkNormalized); %conjP = gpuArray(conj(fftn(chunkNormalized))); %conjPSquared = gpuArray(conj(fftn(chunkNormalized.^2))); %clear chunkNormalized; diff --git a/dynamo/matlab/src/+dpktomo/+match/+aux/ccCore.m b/dynamo/matlab/src/+dpktomo/+match/+aux/ccCore.m index 6e1f280..d31a5ab 100755 --- a/dynamo/matlab/src/+dpktomo/+match/+aux/ccCore.m +++ b/dynamo/matlab/src/+dpktomo/+match/+aux/ccCore.m @@ -18,7 +18,19 @@ %clear Nmask; %norm_data_moving_mask = sqrt(mcn_joint); %clear mcn_joint; -mcn_final = sqrt(ifftn(fftn(rotatedMaskPadded) .* conj(fftn(chunkNormalized.^2))) - 1 * (ifftn(fftn(rotatedMaskPadded) .* conj(fftn(chunkNormalized))).^2) / (Nmask^1)) * norm_rotatedTemplate; +rotatedMaskPadded_ft = fftn(rotatedMaskPadded); +chunkNormalized_sq_ft = fftn(chunkNormalized.^2); +chunkNormalized_ft = fftn(chunkNormalized); +%P11 = arrayfun(@(x,y)(x.*y),rotatedMaskPadded_ft,conj(chunkNormalized_sq_ft)); +P1 = rotatedMaskPadded_ft .* conj(chunkNormalized_sq_ft); +clear chunkNormalized_sq_ft; +%P12 = arrayfun(@(x,y)(x.*y),rotatedMaskPadded_ft,conj(chunkNormalized_ft)); +P2 = rotatedMaskPadded_ft .* conj(chunkNormalized_ft); +clear rotatedMaskPadded_ft; +mcn_final = sqrt(ifftn(P1) - 1 * (ifftn(P2).^2) / (Nmask^1)) * norm_rotatedTemplate; +clear P1; +clear P2; +%mcn_final = sqrt(ifftn(fftn(rotatedMaskPadded) .* conj(fftn(chunkNormalized.^2))) - 1 * (ifftn(fftn(rotatedMaskPadded) .* conj(fftn(chunkNormalized))).^2) / (Nmask^1)) * norm_rotatedTemplate; %mcn_final = real(sqrt(complex(ifftn(fftn(rotatedMaskPadded) .* conj(fftn(chunkNormalized.^2)))) - 1 * (real(ifftn(fftn(rotatedMaskPadded) .* conj(fftn(chunkNormalized)))).^2) / (Nmask^1))) * norm_rotatedTemplate; % clear norm_data_moving_mask; %clear norm_rotatedTemplate ; @@ -32,7 +44,13 @@ % clear P1; % identify the elements that would produce singularities in the numerator % indproblem=find(mcn_final==0); -mc = real(ifftn(fftn(rotatedTemplatePadded).*conj(fftn(chunkNormalized)))./mcn_final); +rotatedTemplatePadded_ft = fftn(rotatedTemplatePadded); +P3 = rotatedTemplatePadded_ft.*conj(chunkNormalized_ft); +P3_ift = ifftn(P3); +clear P3; +mc = P3_ift./mcn_final; +clear P3_ift; +mc = real(mc); % clear RD; mc(mcn_final == 0) = -1; diff --git a/environment/concatAndAddPathsRecursive.m b/environment/concatAndAddPathsRecursive.m index e1a6902..42ea764 100644 --- a/environment/concatAndAddPathsRecursive.m +++ b/environment/concatAndAddPathsRecursive.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function concatAndAddPathsRecursive(path, sub_path_list, separator, debug) if nargin == 2 separator = string(filesep); diff --git a/environment/initializeEnvironment.m b/environment/initializeEnvironment.m index bff11d8..3a4c3dd 100644 --- a/environment/initializeEnvironment.m +++ b/environment/initializeEnvironment.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function initializeEnvironment(default_configuration_path) if ~isdeployed() addpath("configuration"); diff --git a/images/TomoBEAR_file_structures.png b/images/TomoBEAR_file_structures.png new file mode 100755 index 0000000..080c9c9 Binary files /dev/null and b/images/TomoBEAR_file_structures.png differ diff --git a/json/JSON.m b/json/JSON.m index c20f871..f866bff 100644 --- a/json/JSON.m +++ b/json/JSON.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef JSON < handle properties end diff --git a/modules/AreTomo.m b/modules/AreTomo.m index d6fd662..c8a607c 100644 --- a/modules/AreTomo.m +++ b/modules/AreTomo.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - % https://drive.google.com/drive/folders/1Z7pKVEdgMoNaUmd_cOFhlt-QCcfcwF3_ classdef AreTomo < Module methods @@ -57,8 +56,8 @@ % substacks or pseudo subtomograms are aligned? % TODO:NOTE: implement reconstruct aligned tilt series - - min_and_max_tilt_angles = getTiltAngles(obj.configuration); + + min_and_max_tilt_angles = getTiltAngles(obj.configuration, true); % if false == true %length(min_and_max_tilt_angles) > 2 fid = fopen(obj.output_path + filesep + obj.name + ".rawtlt", "w+"); for i = 1:length(min_and_max_tilt_angles) @@ -73,24 +72,46 @@ fprintf(fid, "%s\n", num2str(min_and_max_tilt_angles(i))); end end - angular_file_command_snippet = "-AngFile " + obj.output_path + filesep + obj.name + ".rawtlt"; + angular_file_command_snippet = " -AngFile " + obj.output_path + filesep + obj.name + ".rawtlt"; % else %angle_command_snippet = "-TiltRange " + min_and_max_tilt_angles(1) + " " + min_and_max_tilt_angles(end); - if obj.configuration.patch ~= "0 0" - local_ali = true; - patch_alignment_command_snippet = "-Patch " + strjoin(obj.configuration.patch," "); + if obj.configuration.use_previous_alignment == true + aln_file_path = getFilesFromLastModuleRun(obj.configuration,"AreTomo","aln","last"); + alignment_command_snippet = angular_file_command_snippet... + + " -AlnFile " + string(aln_file_path); else - local_ali = false; - patch_alignment_command_snippet = ""; - end - - tilt_axis_command_snippet = "-TiltAxis " + num2str(obj.configuration.rotation_tilt_axis) + " " + num2str(obj.configuration.tilt_axis_refine_flag); - - if obj.configuration.apply_given_tilt_axis_offset == true - tilt_axis_offset_command_snippet = "-TiltCor " + num2str(obj.configuration.correct_tilt_axis_offset) + " " + num2str(obj.configuration.tilt_axis_offset); - else - tilt_axis_offset_command_snippet = ""; + if obj.configuration.patch ~= "0 0" + local_ali = true; + patch_alignment_command_snippet = " -Patch " + strjoin(obj.configuration.patch," "); + else + local_ali = false; + patch_alignment_command_snippet = ""; + end + + tilt_axis_command_snippet = " -TiltAxis " + num2str(obj.configuration.rotation_tilt_axis) + " " + num2str(obj.configuration.tilt_axis_refine_flag); + + if obj.configuration.correct_pretilt_flag ~= -1 + tilt_axis_offset_command_snippet = " -TiltCor " + num2str(obj.configuration.correct_pretilt_flag); + pretilt_angle_to_use = -obj.configuration.pretilt_tilt_axis; + if pretilt_angle_to_use == 0 + tilt_axis_offset_command_snippet = tilt_axis_offset_command_snippet + " " + num2str(pretilt_angle_to_use); + end + else + tilt_axis_offset_command_snippet = ""; + end + + if obj.configuration.align_z > 0 + align_z_command_snippet = " -AlignZ " + num2str(obj.configuration.align_z); + else + align_z_command_snippet = ""; + end + + alignment_command_snippet = angular_file_command_snippet... + + tilt_axis_command_snippet... + + tilt_axis_offset_command_snippet... + + align_z_command_snippet... + + patch_alignment_command_snippet; end if obj.configuration.set_up.gpu > 0 @@ -112,18 +133,17 @@ + " -InMrc " + stack_source... + " -OutMrc " + stack_destination... + " -VolZ 0 -OutBin " + num2str(obj.configuration.aligned_stack_binning / obj.configuration.input_stack_binning)... - + " -Gpu " + num2str(gpu_number) + " "... - + angular_file_command_snippet + " "... - + tilt_axis_command_snippet + " "... - + tilt_axis_offset_command_snippet + " "... - + patch_alignment_command_snippet... + + " -Gpu " + num2str(gpu_number)... + + alignment_command_snippet... + " -OutImod 1", false, obj.log_file_id); % NOTE: rename ALN file - [~,filename,fileext] = fileparts(stack_source); - alignment_file_out = obj.output_path + filesep + filename + fileext + ".aln"; - alignment_file_destination = obj.output_path + filesep + obj.name + ".aln"; - movefile(alignment_file_out, alignment_file_destination); + if obj.configuration.use_previous_alignment == false + [~,filename,fileext] = fileparts(stack_source); + alignment_file_out = obj.output_path + filesep + filename + fileext + ".aln"; + alignment_file_destination = obj.output_path + filesep + obj.name + ".aln"; + movefile(alignment_file_out, alignment_file_destination); + end % TODO: check linkage here if obj.configuration.aligned_stack_binning == 1 @@ -177,9 +197,11 @@ stack_source = string(tilt_stacks(obj.configuration.set_up.adjusted_j).folder) + string(filesep) + string(tilt_stacks(obj.configuration.set_up.adjusted_j).name); stack_destination_unbinned = obj.output_path + filesep + obj.name + "_bin_" + num2str(1) + ".ali"; - aln_file = dir(obj.output_path + filesep + obj.name + ".aln"); - aln_file_path = aln_file(1).folder + string(filesep) + aln_file(1).name; - + if obj.configuration.use_previous_alignment == false + aln_file = dir(obj.output_path + filesep + obj.name + ".aln"); + aln_file_path = aln_file(1).folder + string(filesep) + aln_file(1).name; + end + executeCommand(obj.configuration.aretomo_command... + " -InMrc " + stack_source... + " -OutMrc " + stack_destination_unbinned... @@ -205,7 +227,7 @@ end folder_destination = obj.configuration.aligned_tilt_stacks_folder; - filename_link_destination = obj.name + "_bin_" + num2str(1) + ".ali"; + filename_link_destination = obj.name + ".ali"; path_destination = obj.configuration.processing_path + filesep + obj.configuration.output_folder + filesep + folder_destination + filesep + obj.name; link_destination = path_destination + filesep + filename_link_destination; @@ -453,6 +475,7 @@ obj.deleteFilesOrFolders(files); obj.deleteFolderIfEmpty(folder); end + obj = cleanUp@Module(obj); end end end diff --git a/modules/BatchRunTomo.m b/modules/BatchRunTomo.m index 7d73b20..8cd38ef 100644 --- a/modules/BatchRunTomo.m +++ b/modules/BatchRunTomo.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - % NOTE: https://bio3d.colorado.edu/imod/betaDoc/alignframesGuide.html % NOTE: https://bio3d.colorado.edu/imod/download.html classdef BatchRunTomo < Module diff --git a/modules/BinStacks.m b/modules/BinStacks.m index 9c102a0..8d8645d 100644 --- a/modules/BinStacks.m +++ b/modules/BinStacks.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef BinStacks < Module methods function obj = BinStacks(configuration) @@ -173,7 +172,12 @@ % end else bin_factor = obj.configuration.binnings(i); - tilt_stacks = getTiltStacks(obj.configuration, true); + if obj.configuration.use_ctf_corrected_aligned_stack == true || obj.configuration.use_aligned_stack == true + tilt_stacks = getAlignedTiltStacksFromStandardFolder(obj.configuration, true); + else + tilt_stacks = getTiltStacks(obj.configuration, true); + end + tilt_stacks = tilt_stacks(contains({tilt_stacks(:).folder}, sprintf("tomogram_%03d", obj.configuration.set_up.j))); if isempty(tilt_stacks) @@ -184,17 +188,22 @@ [path, name, extension] = fileparts(tilt_stacks.name); if obj.configuration.use_ctf_corrected_aligned_stack == true || obj.configuration.use_aligned_stack == true - xf_file_source = getXfOrAlnFilePaths(obj.configuration, obj.output_path, obj.name); - xf_file_destination = obj.output_path + string(filesep) + name + ".xf"; - obj.temporary_files(end + 1) = createSymbolicLink(xf_file_source, xf_file_destination, obj.log_file_id); - xform_command_snippet = " -xform " + xf_file_destination; + stack_source = tilt_stacks.folder + string(filesep) + tilt_stacks.name; + + %xf_file_source = getXfOrAlnFilePaths(obj.configuration, obj.output_path, obj.name); + %xf_file_destination = obj.output_path + string(filesep) + name + ".xf"; + %obj.temporary_files(end + 1) = createSymbolicLink(xf_file_source, xf_file_destination, obj.log_file_id); + %xform_command_snippet = " -xform " + xf_file_destination; + stk_bin_ext = ".ali"; + size_command_snippet = ""; else - xform_command_snippet = ""; + stack_source = tilt_stacks.folder + string(filesep) + tilt_stacks.name; + %xform_command_snippet = ""; stk_bin_ext = ".st"; + size_command_snippet = " -size " + floor(height / (obj.configuration.binnings(i) / obj.configuration.ft_bin)) + "," + floor(width / (obj.configuration.binnings(i) / obj.configuration.ft_bin)); end - stack_source = tilt_stacks.folder + string(filesep) + tilt_stacks.name; stack_destination = obj.output_path + string(filesep) + name + ".st"; obj.temporary_files(end + 1) = createSymbolicLink(stack_source, stack_destination, obj.log_file_id); @@ -208,11 +217,12 @@ [width, height, z] = getHeightAndWidthFromHeader(stack_destination, -1); executeCommand("newstack"... - + " -size " + floor(height / (obj.configuration.binnings(i) / obj.configuration.ft_bin)) + "," + floor(width / (obj.configuration.binnings(i) / obj.configuration.ft_bin))... + + size_command_snippet... + ...%+ " -size " + floor(height / (obj.configuration.binnings(i) / obj.configuration.ft_bin)) + "," + floor(width / (obj.configuration.binnings(i) / obj.configuration.ft_bin))... + " -input " + stack_destination... + " -output " + stack_output_path... ...%+ " -xform " + xf_file_destination... - + xform_command_snippet... + ...%+ xform_command_snippet... + " -antialias " + obj.configuration.antialias_filter... + " -bin " + num2str(bin_factor), false, obj.log_file_id); executeCommand("alterheader -del " + apix + "," + apix + "," + apix + "," + " " + stack_output_path, false, obj.log_file_id); diff --git a/modules/CreateStacks.m b/modules/CreateStacks.m index de77e0c..915aed9 100644 --- a/modules/CreateStacks.m +++ b/modules/CreateStacks.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef CreateStacks < Module methods function obj = CreateStacks(configuration) diff --git a/modules/CryoCARE.m b/modules/CryoCARE.m index 72ebe25..5ba6591 100644 --- a/modules/CryoCARE.m +++ b/modules/CryoCARE.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - % NOTE: https://arxiv.org/pdf/1810.05420.pdf % NOTE: https://github.com/juglab/cryoCARE_pip diff --git a/modules/DLToolModuleTemplate.m b/modules/DLToolModuleTemplate.m new file mode 100644 index 0000000..a9b2c05 --- /dev/null +++ b/modules/DLToolModuleTemplate.m @@ -0,0 +1,193 @@ +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% PUT A LICENSE NOTE HERE +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +classdef DLToolModuleTemplate < Module + methods + %% Module instance constructor + function obj = DLToolModuleTemplate(configuration) + obj@Module(configuration); + end + + %% Pre-execution module setup + function obj = setUp(obj) + obj = setUp@Module(obj); + end + + %% Module execution (main method) + function obj = process(obj) + + return_path = cd(obj.output_path); + + % Get sub-steps of the module to execute + % (e.g. preproc, train, predict, postproc) + steps_to_execute_fields = fieldnames(obj.configuration.steps_to_execute); + + if isempty(steps_to_execute_fields) + error("ERROR: No steps_to_execute were found in the JSON file!"); + end + + % Execute substeps using the corresponding methods + % (methods should be named the same way, as the corresponding + % subsections of the module in config file) + for subjob_idx=1:length(steps_to_execute_fields) + disp("INFO: substep " + string(subjob_idx) + ":" + string(steps_to_execute_fields{subjob_idx})); + step_params = mergeConfigurations(obj.configuration.steps_to_execute_defaults.(steps_to_execute_fields{subjob_idx}),... + obj.configuration.steps_to_execute.(steps_to_execute_fields{subjob_idx}), ... + "DLToolModuleTemplate", "dynamic"); + obj.(steps_to_execute_fields{subjob_idx})(steps_to_execute_fields{subjob_idx}, step_params); + disp("INFO: Execution for substep " + string(subjob_idx) +" (" + string(steps_to_execute_fields{subjob_idx}) + ") has finished!"); + end + + cd(return_path); + end + + %% Method for substeps functionality + + % Method for step_name1 + function obj = step_name1(obj, step_name, step_params) + + % Example of retrieving data: get ctf-corrected tomograms + tomograms = getCtfCorrectedBinnedTomogramsFromStandardFolder(obj.configuration, true); + if isempty(tomograms) + error("ERROR: no ctf corrected tomograms were found!"); + end + + % Example of tomograms filtering by binning level + % take into account binnings parameter from "general" section + % or use specified in config tomograms_binning if requested + if step_params.tomograms_binning == -1 + binnings = sort(obj.configuration.binnings, "descend"); + tomograms_all = tomograms; + for bin_idx=1:length(binnings) + tomograms = tomograms_all(contains({tomograms_all.name}, "bin_" + num2str(binnings(bin_idx)))); + if ~isempty(tomograms) + binning = binnings(bin_idx); + break + end + end + tomograms = tomograms_all(contains({tomograms.name}, "bin_" + binning)); + elseif step_params.tomograms_binning > 1 + binning = step_params.tomograms_binning; + tomograms = tomograms(contains({tomograms.name}, "bin_" + binning)); + end + + if isempty(tomograms) + error("ERROR: no tomograms with the specified binning of were found!"); + end + + % Example of tomograms filtering by selected indeces + % use all tomograms if tomograms_to_use list is empty + % otherwise use only the specified tomograms + if ~isempty(obj.configuration.tomograms_to_use) + tomograms_to_star_str = string(arrayfun(@(a)num2str(a, '%03.f'),obj.configuration.tomograms_to_use,'uni',0)); + tomograms = tomograms(contains(string({tomograms(:).name}), "tomogram_"+tomograms_to_star_str)); + + if isempty(tomograms) + error("ERROR: no tomograms with the specified indices were found!"); + end + end + + % Example of parameters extraction from step_params structure + use_params_cell = {'param1','param2', 'param3'}; + use_params = obj.getRequestedOnlyParametersStructure(step_params, use_params_cell); + + % Example of retrieving a list + % of the requested CPUs number among available ones + if step_params.ncpu == -1 + use_params.ncpu = getCpuPoolSize(1); + else + use_params.ncpu = getCpuPoolSize(1, step_params.ncpu); + end + + % Example of retrieving a list + % of the requested GPUs indeces among available ones + if obj.configuration.gpu > 0 + step_params.gpuID = obj.configuration.gpu - 1; + else + error("ERROR: Gpus are needed for step_name1 step!"); + end + + % Example of initializing absolute path to the requested + % directory to be created in the module/step folder + use_params.output_dir = obj.output_path + string(filesep) + use_params.output_dir; + + % Example of preparing requested parameters string + % and passing it to the DL tool-executing method + params_string = obj.getParamsString(use_params); + params_string = step_name + " " + step_params.star_file + params_string; + % for the case #1 (below) of executeDLToolCommand(): + command_output = obj.executeDLToolCommand(params_string); + % or, if you need to run specific script from the DL tool + % to perform functionality of this particular substep, + % i.e. for the case #2 (below) of executeDLToolCommand(): + command_output = obj.executeDLToolCommand(params_string, tool_script); + end + + %% Auxilary methods to prepare and run external Python DL tool + + % Method to extract initialized parameters of the substep + function parameters_req = getRequestedOnlyParametersStructure(obj, parameters_all, parameters_req_names) + parameters_req = struct(); + for idx=1:length(parameters_req_names) + param_value = parameters_all.(parameters_req_names{idx}); + if (isstring(param_value) && param_value ~= "")... + || (isnumeric(param_value) && isscalar(param_value) && param_value ~= -1) + parameters_req.(parameters_req_names{idx}) = param_value; + elseif isnumeric(param_value) && ~isscalar(param_value) && ~isempty(param_value) + parameters_req.(parameters_req_names{idx}) = strjoin(string(param_value), ","); + end + end + end + + % Method to convert parameters values to the parameters string + function params_string = getParamsString(obj, params, param_prefix) + if nargin < 3 + param_prefix="--"; + end + params_fields = fieldnames(params); + params_string = ""; + for idx=1:length(params_fields) + params_string = params_string + " " + param_prefix + params_fields{idx} + " " + params.(params_fields{idx}); + end + end + + % Methods to execute external Python DL tool + + % Case #1: DL tool is cloned from GitHub repository + % (with dependencies being installed by user in the separate conda env) + % Example tool: IsoNet + function command_output = executeDLToolCommand(obj, params_string) + python_run_script_snippet = "PYTHONPATH=" + fullfile(obj.configuration.repository_path, '..'); + + % Add to command string non-interactive conda environment call + if obj.configuration.use_conda == true + python_run_script_snippet = python_run_script_snippet + " LD_LIBRARY_PATH=" + obj.configuration.conda_path + filesep + "lib:$LD_LIBRARY_PATH conda run -n " + obj.configuration.dltool_env; + end + + python_run_script_snippet = python_run_script_snippet + " python " + obj.configuration.repository_path + filesep + "/path/to/main_file.py"; + + % Merge params string with command string and execute target + command_output = executeCommand(python_run_script_snippet + " " + params_string, false, obj.log_file_id); + end + + % Case #2: DL tool is installed via conda as a package + % (with dependencies coming along with the tool in the same conda env) + % Example tool: crYOLO + function command_output = executeDLToolCommand(obj, params_string, tool_script) + + % Add to command string non-interactive conda environment call + if obj.configuration.use_conda == true + python_run_script_snippet = python_run_script_snippet + " LD_LIBRARY_PATH=" + obj.configuration.conda_path + filesep + "lib:$LD_LIBRARY_PATH conda run -n " + obj.configuration.dltool_env; + end + command_snippet = obj.configuration.conda_path + string(filesep) + "envs"... + + string(filesep) + obj.configuration.dltool_env + string(filesep) + "bin"... + + string(filesep) + tool_script; + python_run_script_snippet = python_run_script_snippet + " python " + command_snippet; + + % Merge params string with command string and execute target + command_output = executeCommand(python_run_script_snippet + " " + params_string, false, obj.log_file_id); + end + end +end + diff --git a/modules/DeepFinder.m b/modules/DeepFinder.m index bec2624..a85c0d1 100644 --- a/modules/DeepFinder.m +++ b/modules/DeepFinder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef DeepFinder < Module methods function obj = DeepFinder(configuration) diff --git a/modules/DynamoAlignmentProject.m b/modules/DynamoAlignmentProject.m index 15a1b90..79094fd 100644 --- a/modules/DynamoAlignmentProject.m +++ b/modules/DynamoAlignmentProject.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef DynamoAlignmentProject < Module methods diff --git a/modules/DynamoCleanStacks.m b/modules/DynamoCleanStacks.m index 4d7af6a..8ce3556 100644 --- a/modules/DynamoCleanStacks.m +++ b/modules/DynamoCleanStacks.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef DynamoCleanStacks < Module methods function obj = DynamoCleanStacks(configuration) diff --git a/modules/DynamoImportTomograms.m b/modules/DynamoImportTomograms.m index a6652ea..20a208c 100644 --- a/modules/DynamoImportTomograms.m +++ b/modules/DynamoImportTomograms.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef DynamoImportTomograms < Module methods function obj = DynamoImportTomograms(configuration) diff --git a/modules/DynamoTemplateMatching.m b/modules/DynamoTemplateMatching.m index 5dbe697..8a363fc 100644 --- a/modules/DynamoTemplateMatching.m +++ b/modules/DynamoTemplateMatching.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef DynamoTemplateMatching < Module methods function obj = DynamoTemplateMatching(configuration) @@ -38,7 +37,15 @@ return_path = cd(dynamo_folder); %dynamo_folder = obj.configuration.processing_path + string(filesep) + obj.configuration.output_folder + string(filesep) + obj.configuration.dynamo_folder; - if isfield(obj.configuration, "use_ctf_corrected_tomograms") && obj.configuration.use_ctf_corrected_tomograms == true + if isfield(obj.configuration, "use_custom_tomograms_path") && isfolder(obj.configuration.use_custom_tomograms_path) + tomograms = dir(obj.configuration.use_custom_tomograms_path + string(filesep) + "*.rec"); + if isempty(tomograms) + tomograms = dir(obj.configuration.use_custom_tomograms_path + string(filesep) + "*.mrc"); + if isempty(tomograms) + error("ERROR: no tomograms were found! Provided path: " + obj.configuration.use_custom_tomograms_path); + end + end + elseif isfield(obj.configuration, "use_ctf_corrected_tomograms") && obj.configuration.use_ctf_corrected_tomograms == true if isfield(obj.configuration, "use_denoised_tomograms") && obj.configuration.use_denoised_tomograms == true if isfield(obj.configuration, "template_matching_binning") && obj.configuration.template_matching_binning > 1 tomograms = getDenoisedCtfCorrectedBinnedTomogramsFromStandardFolder(obj.configuration, true, obj.configuration.template_matching_binning); diff --git a/modules/DynamoTiltSeriesAlignment.m b/modules/DynamoTiltSeriesAlignment.m index cd34180..d92ae39 100644 --- a/modules/DynamoTiltSeriesAlignment.m +++ b/modules/DynamoTiltSeriesAlignment.m @@ -1,23 +1,22 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef DynamoTiltSeriesAlignment < Module methods function obj = DynamoTiltSeriesAlignment(configuration) diff --git a/modules/EMDTemplateGeneration.m b/modules/EMDTemplateGeneration.m index 5fa84d5..0dbb415 100644 --- a/modules/EMDTemplateGeneration.m +++ b/modules/EMDTemplateGeneration.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef EMDTemplateGeneration < Module methods function obj = EMDTemplateGeneration(configuration) diff --git a/modules/GCTFCtfphaseflipCTFCorrection.m b/modules/GCTFCtfphaseflipCTFCorrection.m index 5469207..3fe6c92 100644 --- a/modules/GCTFCtfphaseflipCTFCorrection.m +++ b/modules/GCTFCtfphaseflipCTFCorrection.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef GCTFCtfphaseflipCTFCorrection < Module methods function obj = GCTFCtfphaseflipCTFCorrection(configuration) diff --git a/modules/GainCorrection.m b/modules/GainCorrection.m index bfc8e25..018ef9a 100644 --- a/modules/GainCorrection.m +++ b/modules/GainCorrection.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - % NOTE:TODO: relion_estimate_gain classdef GainCorrection < Module diff --git a/modules/GeneralModuleTemplate.m b/modules/GeneralModuleTemplate.m new file mode 100644 index 0000000..6367df7 --- /dev/null +++ b/modules/GeneralModuleTemplate.m @@ -0,0 +1,135 @@ +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% PUT A LICENSE NOTE HERE +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +classdef GeneralModuleTemplate < Module + methods + %% Module instance constructor + function obj = GeneralModuleTemplate(configuration) + obj@Module(configuration); + end + + %% Pre-execution module setup + function obj = setUp(obj) + obj = setUp@Module(obj); + + % Create data_folder or metadata_folder in project folder + createStandardFolder(obj.configuration, "data_folder", false); + createStandardFolder(obj.configuration, "metadata_folder", false); + end + + %% Module execution (main method) + function obj = process(obj) + %% 1. Get input data + % > Get files/directories/parameters from global configuration + % > Calculate and set necessary temporal configuration params + + % Get global configuration parameter PARAM (which must be present in configuration anyway) + config_param = obj.configuration.PARAM; + + % Use global configuration parameter PARAM (which could not be present in configuration) + if isfield(obj.configuration, "PARAM") && CHECK_IF_EMPTY + % CHECK_IF_EMPTY depends on data type, alternatives are: + % if PARAM is a number: obj.configuration.PARAM ~= -1 + % if PARAM is a string: obj.configuration.PARAM ~= "" + % if PARAM is a list: ~isempty(obj.configuration.PARAM) + + % put here code accessing obj.configuration.PARAM + end + + % Set dynamic configuration parameter PARAM with VALUE + obj.dynamic_configuration.PARAM = VALUE; + + % Get tomogram titles + field_names = fieldnames(obj.configuration.tomograms); + + % Access tomogram parameter PARAM by its title index IDX + data = obj.configuration.tomograms.(field_names{IDX}).PARAM; + + % Get files of the certain type from previous steps + filepaths = getFilesFromLastModuleRun(obj.configuration, "GeneralModuleTemplate", "extension", "last"); + % or use other ways to get specific files, e.g. check folder + % utilities/ for available functions or add necessary + % functionality right as a new utilities/ function or + % as an additional method of the current module (see below) + + %% 2. Execute target module code + % > Use your original algorithm + % by inserting it directly here or by using the custom method + % calling it with some INPUTS set and geting OUTPUTS set as + [OUTPUTS] = obj.customMethod(INPUTS); + + % > Wrap external (CLI) software call + + % If you want to code module to be parallelized + % on CPUs using "parallel" execution method, or + % on GPUs using "in_order" execution method + % the assigned index of a parallelized piece of data + % (currently only tilt serie/tomogram, but can be else) + % to be processed on the assigned CPU or GPU worker is: + % obj.configuration.set_up.adjusted_j + % To access the PARAM parameter of the corresponding + % computing resource-assigned tomogram use + data = obj.configuration.tomograms.(field_names{obj.configuration.set_up.adjusted_j}).PARAM; + + % Additionaly, for GPU-parallelized module execution + % (execution_method == "in_order") to get assigned GPU use: + if obj.configuration.set_up.gpu > 0 + gpu_number = obj.configuration.set_up.gpu - 1; + else + error("ERROR: Gpus are needed to run this module!"); + end + + % For a general external tool_command launch + % for all the available (data processing) execution methods + % (i.e. "once", "sequential", "parallel", "in_order") + executeCommand(obj.configuration.tool_command... + + " -arg1 " + num2str(argument_value_number)... + + " -arg2 " + strjoin(num2str(argument_value_numbers_list), ",")... + ... % some other parameters + + " -argN " + argument_value_string, false, obj.log_file_id); + % the last parameter is to write command output to the log file + + %% 3. Link output data and metadata to standard locations + + % Make symlink using source file and destination file paths + createSymbolicLink(source_file_path, dest_file_path, obj.log_file_id); + + % Make symlink using source file and destination dir paths + % where destination dir is a (meta)data_folder in project dir + createSymbolicLinkInStandardFolder(obj.configuration, source_file_path, "data_folder", obj.log_file_id); + + %% N.B. Inform user about execution status by INFO and ERROR + + % Display info-message for user during execution + disp("INFO: Put some info message here..."); + + % Display error message for user during execution + error("ERROR: Put some error message here!"); + + end + + %% Custom processing method + function [outputs] = customMethod(obj, inputs) + % Put your custom method code here + end + + %% Post-execution routines + function obj = cleanUp(obj) + % Automated data removal (execution modes except "cleanup") + % > for files with folders + obj.deleteFilesOrFolders(files); + % > for folders only + obj.deleteFolderIfEmpty(folder); + + % Automated data removal ("cleanup" execution mode only) + if obj.configuration.execute == false && obj.configuration.keep_intermediates == false + % put here your code for cleanup + end + + % Post-execution routines (execution modes except "cleanup") + % write TIME and step log files + obj = cleanUp@Module(obj); + end + end +end \ No newline at end of file diff --git a/modules/GenerateParticles.m b/modules/GenerateParticles.m index b1b368c..d531781 100644 --- a/modules/GenerateParticles.m +++ b/modules/GenerateParticles.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef GenerateParticles < Module methods function obj = GenerateParticles(configuration) diff --git a/modules/GridEdgeEraser.m b/modules/GridEdgeEraser.m index d0d0a3d..4f9c282 100644 --- a/modules/GridEdgeEraser.m +++ b/modules/GridEdgeEraser.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef GridEdgeEraser < Module methods function obj = GridEdgeEraser(configuration) diff --git a/modules/IsoNet.m b/modules/IsoNet.m index 199ef5c..a663364 100644 --- a/modules/IsoNet.m +++ b/modules/IsoNet.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - % NOTE: https://github.com/IsoNet-cryoET/IsoNet classdef IsoNet < Module @@ -71,6 +70,7 @@ break end end + tomograms = tomograms_all(contains({tomograms.name}, "bin_" + binning)); elseif step_params.tomograms_binning > 1 binning = step_params.tomograms_binning; tomograms = tomograms(contains({tomograms.name}, "bin_" + binning)); @@ -325,6 +325,31 @@ command_output = obj.executeIsoNetCommand(params_string); end + function obj = postprocess(obj, step_name, step_params) + + step_params.i = step_params.nad_filter_output_iterations_list; + step_params.n = step_params.nad_filter_number_of_iterations; + step_params.s = step_params.nad_filter_sigma_for_smoothing; + step_params.k = step_params.nad_filter_threshold_for_gradients; + + use_params_cell = {'i', 'n', 's', 'k'}; + use_params = obj.getRequestedOnlyParametersStructure(step_params, use_params_cell); + params_string = obj.getParamsString(use_params, "-"); + params_string = "nad_eed_3d" + params_string; + + step_params.input_dir = obj.output_path + string(filesep) + step_params.input_dir; + step_params.output_dir = obj.output_path + string(filesep) + step_params.output_dir; + mkdir(step_params.output_dir); + input_tomograms = dir(step_params.input_dir + filesep + '*.mrc'); + for file_id = 1:length(input_tomograms) + input_tomogram_path = step_params.input_dir + string(filesep) + input_tomograms(file_id).name; + [~,filename,fileext] = fileparts(input_tomograms(file_id).name); + output_tomogram_path = step_params.output_dir + string(filesep) + filename + "_nadf" + fileext; + cmd_string = params_string + " " + input_tomogram_path + " " + output_tomogram_path; + output_msg = executeCommand(cmd_string, false, obj.log_file_id); + end + end + function parameters_req = getRequestedOnlyParametersStructure(obj, parameters_all, parameters_req_names) parameters_req = struct(); for idx=1:length(parameters_req_names) @@ -338,11 +363,14 @@ end end - function params_string = getParamsString(obj, params) + function params_string = getParamsString(obj, params, param_prefix) + if nargin < 3 + param_prefix="--"; + end params_fields = fieldnames(params); params_string = ""; for idx=1:length(params_fields) - params_string = params_string + " --" + params_fields{idx} + " " + params.(params_fields{idx}); + params_string = params_string + " " + param_prefix + params_fields{idx} + " " + params.(params_fields{idx}); end end diff --git a/modules/MetaData.m b/modules/MetaData.m index 1c218ef..04ef3a7 100644 --- a/modules/MetaData.m +++ b/modules/MetaData.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef MetaData < Module methods function obj = MetaData(configuration) diff --git a/modules/Module.m b/modules/Module.m index ab8e58c..c80728b 100644 --- a/modules/Module.m +++ b/modules/Module.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef (Abstract) Module properties configuration struct; diff --git a/modules/MotionCor2.m b/modules/MotionCor2.m index cb6fa48..dba5f37 100644 --- a/modules/MotionCor2.m +++ b/modules/MotionCor2.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef MotionCor2 < Module methods function obj = MotionCor2(configuration) @@ -141,9 +140,12 @@ end if extension == ".eer" + data_is_eer = true; + eer_upsampling_factor = 2 ^ (obj.configuration.eer_upsampling - 1); motion_correction_arguments = motion_correction_arguments + " "... - + " -PixSize " + (obj.configuration.tomograms.(field_names{obj.configuration.set_up.j}).apix / obj.configuration.eer_sampling); + + " -PixSize " + (obj.configuration.tomograms.(field_names{obj.configuration.set_up.j}).apix / eer_upsampling_factor); else + data_is_eer = false; motion_correction_arguments = motion_correction_arguments + " "... + " -PixSize " + obj.configuration.tomograms.(field_names{obj.configuration.set_up.j}).apix; end @@ -229,7 +231,7 @@ + " -OutAln " + obj.configuration.out_aln; end - if obj.configuration.gain ~= "" + if obj.configuration.dark ~= "" [folder, name, extension] = fileparts(obj.configuration.dark); name_rep = strrep(name, " ", "\ "); dark_path = folder + string(filesep) + name_rep + extension; @@ -348,7 +350,7 @@ + " -InMrc " + input_projection; elseif extension == ".eer" command = obj.configuration.motion_correction_command... - + " -InEer " + input_projection + " -EerSampling " + obj.configuration.eer_sampling; + + " -InEer " + input_projection + " -EerSampling " + obj.configuration.eer_upsampling; if obj.configuration.fm_int_file ~= "" command = command + " -FmIntFile " + obj.configuration.fm_int_file; else @@ -364,17 +366,18 @@ + " " + motion_correction_arguments + " "... + " -LogFile " + motion_correction_log; - if obj.configuration.fm_ref ~= "" + if obj.configuration.fm_ref ~= -1 command = command... + " -FmRef " + obj.configuration.fm_ref; elseif extension == ".eer" command = command... - + " -FmRef " + obj.configuration.eer_total_number_of_fractions / obj.configuration.eer_fraction_grouping; + + " -FmRef " + floor(obj.configuration.eer_total_number_of_fractions / obj.configuration.eer_fraction_grouping); else command = command... + " -FmRef " + last_frame; end + %{ if ~isempty(regexp(input_projection, "_[-+]*0+\.0", "match")) && isfield(obj.configuration.tomograms.(field_names{obj.configuration.set_up.j}), "high_dose") && obj.configuration.tomograms.(field_names{obj.configuration.set_up.j}).high_dose == true command = command... + " -Patch " + obj.configuration.patch; @@ -387,6 +390,15 @@ + " -Group " + string(num2str(obj.configuration.group)); end end + %} + % NOTE: patches and groups crash when used simultaneously + if obj.configuration.patch ~= "" + command = command... + + " -Patch " + obj.configuration.patch; + elseif obj.configuration.group > 1 + command = command... + + " -Group " + string(num2str(obj.configuration.group)); + end if obj.configuration.apply_dose_weighting == true if obj.configuration.tomograms.(field_names{obj.configuration.set_up.j}).high_dose == true @@ -423,6 +435,14 @@ % end motion_corrected_files{i} = mrc_output; end + + % NOTE: Update global ft_bin value to match binning level + % relative to the set in "general" value of apix + if data_is_eer == true + eer_upsampling_factor = 2 ^ (obj.configuration.eer_upsampling - 1); + ft_bin = obj.configuration.ft_bin; + obj.configuration.ft_bin = ft_bin / eer_upsampling_factor; + end end function [motion_corrected_files, symbolic_link_standard_folder] = summarizeOnly(obj, mrc_list) diff --git a/modules/PolishParticlesAndEstimateResolution.m b/modules/PolishParticlesAndEstimateResolution.m index 15b9174..d1a8cbd 100644 --- a/modules/PolishParticlesAndEstimateResolution.m +++ b/modules/PolishParticlesAndEstimateResolution.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef PolishParticlesAndEstimateResolution < Module methods function obj = PolishParticlesAndEstimateResolution(configuration) diff --git a/modules/Reconstruct.m b/modules/Reconstruct.m index a9e72c7..1a6b0b4 100644 --- a/modules/Reconstruct.m +++ b/modules/Reconstruct.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef Reconstruct < Module methods function obj = Reconstruct(configuration) diff --git a/modules/RefineFiducialModel.m b/modules/RefineFiducialModel.m index 3288699..6580a9b 100644 --- a/modules/RefineFiducialModel.m +++ b/modules/RefineFiducialModel.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef RefineFiducialModel < Module methods function obj = RefineFiducialModel(configuration) diff --git a/modules/RelionAlignmentProject.m b/modules/RelionAlignmentProject.m index c787c55..39ff790 100644 --- a/modules/RelionAlignmentProject.m +++ b/modules/RelionAlignmentProject.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef RelionAlignmentProject < Module methods function obj = RelionAlignmentProject(configuration) diff --git a/modules/SortFiles.m b/modules/SortFiles.m index 65b8cb5..94e6136 100644 --- a/modules/SortFiles.m +++ b/modules/SortFiles.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef SortFiles < Module methods function obj = SortFiles(configuration) diff --git a/modules/TemplateGenerationFromFile.m b/modules/TemplateGenerationFromFile.m index 9b8827d..26ed9f5 100644 --- a/modules/TemplateGenerationFromFile.m +++ b/modules/TemplateGenerationFromFile.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef TemplateGenerationFromFile < Module methods function obj = TemplateGenerationFromFile(configuration) diff --git a/modules/TemplateMatchingPostProcessing.m b/modules/TemplateMatchingPostProcessing.m index 3bc1e21..0580f42 100644 --- a/modules/TemplateMatchingPostProcessing.m +++ b/modules/TemplateMatchingPostProcessing.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef TemplateMatchingPostProcessing < Module methods function obj = TemplateMatchingPostProcessing(configuration) @@ -118,9 +117,11 @@ end end - tab_all_path = obj.configuration.processing_path + string(filesep) + obj.configuration.output_folder + string(filesep) + obj.configuration.particles_table_folder; - tab_all_file_path = char(tab_all_path + string(filesep) + "tab_ini_all_bin_" + obj.configuration.template_matching_binning + "_" + num2str(size(tab_all,1)) + ".tbl"); + tab_all_file_path = char(output_path + string(filesep) + "tab_ini_all_bin_" + obj.configuration.template_matching_binning + "_" + num2str(size(tab_all,1)) + ".tbl"); dwrite(tab_all, tab_all_file_path); + + link_destination = obj.configuration.processing_path + string(filesep) + obj.configuration.output_folder + string(filesep) + obj.configuration.particles_table_folder; + createSymbolicLink(tab_all_file_path, link_destination, obj.log_file_id); if obj.configuration.crop_particles == true generateParticles(obj.configuration, tab_all_file_path, obj.configuration.template_matching_binning, box_size, true); diff --git a/modules/TomoAlign.m b/modules/TomoAlign.m index da38eda..2d2ff9b 100644 --- a/modules/TomoAlign.m +++ b/modules/TomoAlign.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef TomoAlign < Module methods function obj = TomoAlign(configuration) diff --git a/modules/crYOLO.m b/modules/crYOLO.m new file mode 100644 index 0000000..ffa3f7c --- /dev/null +++ b/modules/crYOLO.m @@ -0,0 +1,391 @@ +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +% This file is part of the TomoBEAR software. +% Copyright (c) 2021,2022,2023 TomoBEAR Authors +% +% This program is free software: you can redistribute it and/or modify +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. +% +% This program is distributed in the hope that it will be useful, +% but WITHOUT ANY WARRANTY; without even the implied warranty +% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License +% along with this program. If not, see . +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + + +% NOTE: https://cryolo.readthedocs.io/en/stable/index.html + +classdef crYOLO < Module + methods + function obj = crYOLO(configuration) + obj@Module(configuration); + createStandardFolder(obj.configuration, "particles_table_folder"); + end + + function obj = process(obj) + + return_path = cd(obj.output_path); + + steps_to_execute_fields = fieldnames(obj.configuration.steps_to_execute); + + if isempty(steps_to_execute_fields) + error("ERROR: No IsoNet steps_to_execute were found in the JSON file!"); + end + + % NOTE: add 'success' files to be able to re-run substeps + % NOTE: add 'stop' execution control step + for subjob_idx=1:length(steps_to_execute_fields) + disp("INFO: crYOLO substep " + string(subjob_idx) + ":" + string(steps_to_execute_fields{subjob_idx})); + step_params = mergeConfigurations(obj.configuration.steps_to_execute_defaults.(steps_to_execute_fields{subjob_idx}), obj.configuration.steps_to_execute.(steps_to_execute_fields{subjob_idx}), "crYOLO", "dynamic"); + obj.(steps_to_execute_fields{subjob_idx})(step_params); + disp("INFO: Execution for crYOLO substep " + string(subjob_idx) +" (" + string(steps_to_execute_fields{subjob_idx}) + ") has finished!"); + end + + cd(return_path); + end + + function obj = config(obj, step_params) + + use_params_cell = {'filter', 'input_size'}; + if step_params.filter == "LOWPASS" + use_params_cell{length(use_params_cell)+1} = 'low_pass_cutoff'; + elseif step_params.filter == "JANNI" + use_params_cell{length(use_params_cell)+1} = 'janni_model_path'; + end + + if step_params.train_mode == true + use_params_cell{length(use_params_cell)+1} = 'train_image_folder'; + use_params_cell{length(use_params_cell)+1} = 'train_annot_folder'; + + if ~isfolder(step_params.train_tomograms_folder) || isempty(step_params.train_tomograms_folder) + if step_params.train_tomograms_path == "" + obj.LinkTomogramsToRequestedDirectory(step_params.tomograms_binning, step_params.train_tomograms_folder); + else + obj.LinkTomogramsToRequestedDirectory(step_params.tomograms_binning, step_params.train_tomograms_folder, step_params.train_tomograms_path); + end + step_params.train_tomograms_folder = obj.output_path + string(filesep) + step_params.train_tomograms_folder; + step_params.train_image_folder = step_params.train_tomograms_folder; + end + + if ~isfolder(step_params.train_annot_folder) + mkdir(step_params.train_annot_folder); + disp("WARNING: Has not found any annotation files! Be sure to prepare them before the training!"); + step_params.train_annot_folder = obj.output_path + string(filesep) + step_params.train_annot_folder; + end + end + + use_params = obj.getRequestedOnlyParametersStructure(step_params, use_params_cell); + + step_params.config_json_filepath = obj.output_path + string(filesep) + step_params.config_json_filepath; + params_string = obj.getParamsString(use_params); + params_string = step_params.config_json_filepath... + + " " + step_params.target_boxsize... + + " " + params_string; + command_output = obj.executeCrYOLOCommand(params_string, step_params.cryolo_command); + end + + function obj = train(obj, step_params) + + if ~isfile(step_params.config_json_filepath) + error('ERROR: the requested config_json_filepath was not found!'); + elseif isfile(obj.output_path + string(filesep) + step_params.config_json_filepath) + step_params.config_json_filepath = obj.output_path + string(filesep) + step_params.config_json_filepath; + end + + if obj.configuration.gpu > 0 + step_params.gpu = obj.configuration.gpu - 1; + else + error("ERROR: Gpus are needed to train crYOLO!"); + end + + use_params_cell = {'early','warmup','gpu'}; + + use_params = obj.getRequestedOnlyParametersStructure(step_params, use_params_cell, " "); + + if step_params.num_cpu == -1 + use_params.num_cpu = getCpuPoolSize(1); + else + use_params.num_cpu = getCpuPoolSize(1, step_params.num_cpu); + end + + params_string = "-c " + step_params.config_json_filepath + " " + obj.getParamsString(use_params); + command_output = obj.executeCrYOLOCommand(params_string, step_params.cryolo_command); + end + + function obj = predict(obj, step_params) + + if ~isfile(step_params.config_json_filepath) + error('ERROR: the requested config_json_filepath was not found!'); + elseif isfile(obj.output_path + string(filesep) + step_params.config_json_filepath) + step_params.config_json_filepath = obj.output_path + string(filesep) + step_params.config_json_filepath; + end + + if obj.configuration.gpu > 0 + step_params.gpu = obj.configuration.gpu - 1; + else + error("ERROR: Gpus are needed to predict using crYOLO!"); + end + + if ~isfile(step_params.trained_model_filepath) + error('ERROR: the requested trained_model_filepath was not found!'); + elseif isfile(obj.output_path + string(filesep) + step_params.trained_model_filepath) + step_params.weights = obj.output_path + string(filesep) + step_params.trained_model_filepath; + else + step_params.weights = step_params.trained_model_filepath; + end + + if ~isfolder(step_params.test_tomograms_folder) || isempty(step_params.test_tomograms_folder) + if step_params.test_tomograms_path == "" + obj.LinkTomogramsToRequestedDirectory(step_params.tomograms_binning, step_params.test_tomograms_folder); + else + obj.LinkTomogramsToRequestedDirectory(step_params.tomograms_binning, step_params.test_tomograms_folder, step_params.test_tomograms_path); + end + step_params.input = obj.output_path + string(filesep) + step_params.test_tomograms_folder; + elseif isfolder(obj.output_path + string(filesep) + step_params.test_tomograms_folder) + step_params.input = obj.output_path + string(filesep) + step_params.test_tomograms_folder; + else + step_params.input = step_params.test_tomograms_folder; + end + + if isfolder(obj.output_path + string(filesep) + step_params.predict_annot_folder) + step_params.output = obj.output_path + string(filesep) + step_params.predict_annot_folder; + else + step_params.output = step_params.predict_annot_folder; + end + + use_params_cell = {'weights', 'input', 'output', 'threshold',... + 'tracing_search_range','tracing_memory','tracing_min_length',... + 'gpu'}; + + use_params = obj.getRequestedOnlyParametersStructure(step_params, use_params_cell, " "); + + if step_params.num_cpu == -1 + use_params.num_cpu = getCpuPoolSize(1); + else + use_params.num_cpu = getCpuPoolSize(1, step_params.num_cpu); + end + + params_string = "-c " + step_params.config_json_filepath + " " + obj.getParamsString(use_params) + " --tomogram"; + command_output = obj.executeCrYOLOCommand(params_string, step_params.cryolo_command); + end + + function obj = export_annotations(obj, step_params) + + prtcl_coord_path = step_params.raw_prtcl_coords_dir + string(filesep) + "*.coords"; + prtcl_coord_files = dir(prtcl_coord_path); + + if isempty(prtcl_coord_files) + error("ERROR: particles table was requested, but annotated raw particles coordinates were not found!"); + end + + [~, name, ~] = fileparts(prtcl_coord_files(1).name); + name_splitted = strsplit(name, "_"); + tomogram_binning = str2num(name_splitted{4}); + % NOTE: get per each tomo separate particles table + output_path = obj.output_path; + per_table_particle_count = step_params.per_table_particle_count; + tab_tomo = {}; + for i = 1:length(prtcl_coord_files) + prtcl_coord_filepath = prtcl_coord_files(i).folder + string(filesep) + prtcl_coord_files(i).name; + tab_tomo{i} = produceParticlesTableForTomogram(prtcl_coord_filepath, per_table_particle_count, output_path); + end + + % NOTE: merge individual particles tables + tab_all = dynamo_table_blank(size(tab_tomo{1}, 1)); + sum_particles_previous_table = 0; + particle_count = step_params.total_particle_count; + + break_flag = false; + for i = 1:length(tab_tomo) + if isempty(tab_tomo{i}) || tab_tomo{i}(1,20) == 0 + continue; + end + if i == 1 + tab_all(1:end, :) = tab_tomo{i}; + else + tab_tomo_tmp = tab_tomo{i}; + tab_tomo_tmp(:,1) = tab_tomo{i}(:,1) + sum_particles_previous_table; + tab_all(end + 1:end + size(tab_tomo{i}, 1), :) = tab_tomo_tmp; + end + + sum_particles_previous_table = sum_particles_previous_table + size(tab_tomo{i}, 1); + if particle_count > 0 + if sum_particles_previous_table > particle_count + tab_all(particle_count + 1:end, :) = []; + tab_tomo{i}(particle_count + 1:end, :) = []; + break_flag = true; + else + particle_count = particle_count - sum_particles_previous_table; + end + end + + if break_flag == true + break; + end + end + + tab_all_file_path = char(output_path + string(filesep) + "tab_ini_all_bin_" + tomogram_binning + "_" + num2str(size(tab_all,1)) + ".tbl"); + dwrite(tab_all, tab_all_file_path); + + link_destination = obj.configuration.processing_path + string(filesep) + obj.configuration.output_folder + string(filesep) + obj.configuration.particles_table_folder; + createSymbolicLink(tab_all_file_path, link_destination, obj.log_file_id); + + function tab_tomo = produceParticlesTableForTomogram(raw_prtcl_coords_filepath, particle_count, output_path) + + [~, name, ~] = fileparts(raw_prtcl_coords_filepath); + name_splitted = strsplit(name, "_"); + tomogram_number = str2num(name_splitted{2}); + tomogram_binning = str2num(name_splitted{4}); + + if fileExists(output_path + string(filesep) + "SUCCESS_" + tomogram_number) + table_file = dir(output_path + string(filesep)... + + "tab_" + num2str(tomogram_number) + "_ini_bin_"... + + num2str(tomogram_binning) + "_*.tbl"); + table_file_path = table_file(1).folder + string(filesep) + table_file(1).name; + tab_tomo = dread(char(table_file_path)); + else + raw_tab_tomo = readtable(raw_prtcl_coords_filepath, 'FileType', 'text'); + raw_tab_tomo = table2array(raw_tab_tomo); + + if particle_count > 0 + particle_count = min(size(raw_tab_tomo,1), particle_count); + raw_tab_tomo = raw_tab_tomo(1:particle_count,:); + else + particle_count = size(raw_tab_tomo,1); + end + disp("INFO: " + num2str(particle_count) + " particles were found in tomogram_" + num2str(tomogram_number)); + + tab_tomo = dynamo_table_blank(particle_count); + + tab_tomo(1:particle_count,1) = 1:particle_count; + tab_tomo(1:particle_count,2:3) = ones(particle_count,2); + tab_tomo(1:particle_count,24:26) = raw_tab_tomo(:,:); + + tab_tomo(1:particle_count,20) = tomogram_number * ones(particle_count,1); + tab_tomo(1:particle_count,32) = ones(particle_count,1); + + if tab_tomo(1,20) ~= 0 + table_file_path = output_path + string(filesep)... + + "tab_" + num2str(tomogram_number) + "_ini_bin_"... + + num2str(tomogram_binning) + "_"... + + num2str(particle_count) + ".tbl"; + + dwrite(tab_tomo, table_file_path); + + %writetable(tab_tomo, table_file_path, 'WriteVariableNames', false, 'FileType', 'text', 'Delimiter', ' '); + + fid = fopen(output_path + string(filesep)... + + "SUCCESS_" + num2str(tomogram_number), "w"); + fclose(fid); + else + fid = fopen(output_path + string(filesep)... + + "FAILURE_" + num2str(tomogram_number), "w"); + fclose(fid); + end + end + end + end + + function LinkTomogramsToRequestedDirectory(obj, tomograms_binning, tomograms_folder_dest, tomograms_folder_src) + + if nargin < 3 + tomograms = getCtfCorrectedBinnedTomogramsFromStandardFolder(obj.configuration, true); + else + tomograms = dir(tomograms_folder_src + string(filesep) + "*.rec"); + if isempty(tomograms) + tomograms = dir(tomograms_folder_src + string(filesep) + "*.mrc"); + end + end + + if isempty(tomograms) + error("ERROR: no ctf corrected tomograms were found!"); + end + + if tomograms_binning == -1 + binnings = sort(obj.configuration.binnings, "descend"); + tomograms_all = tomograms; + for bin_idx=1:length(binnings) + tomograms = tomograms_all(contains({tomograms_all.name}, "bin_" + num2str(binnings(bin_idx)))); + if ~isempty(tomograms) + binning = binnings(bin_idx); + break + end + end + tomograms = tomograms_all(contains({tomograms.name}, "bin_" + binning)); + elseif tomograms_binning >= 1 + binning = tomograms_binning; + tomograms = tomograms(contains({tomograms.name}, "bin_" + binning)); + else + error("ERROR: the input binning level value is incorrect..."); + end + + if isempty(tomograms) + error("ERROR: no tomograms with the specified binning level were found!"); + end + + if ~isempty(obj.configuration.tomograms_to_use) + tomograms_to_star_str = string(arrayfun(@(a)num2str(a, '%03.f'),obj.configuration.tomograms_to_use,'uni',0)); + tomograms = tomograms(contains(string({tomograms(:).name}), "tomogram_"+tomograms_to_star_str)); + + if isempty(tomograms) + error("ERROR: no tomograms with the specified indices were found!"); + end + end + + % NOTE: add possibility to use external tomograms + tomograms_path = obj.output_path + string(filesep) + tomograms_folder_dest; + mkdir(tomograms_path); + for tomo_idx=1:length(tomograms) + tomogram_source = tomograms(tomo_idx).folder + string(filesep) + tomograms(tomo_idx).name; + link_destination = tomograms_path + string(filesep) + tomograms(tomo_idx).name; + createSymbolicLink(tomogram_source, link_destination, obj.log_file_id); + end + end + + function parameters_req = getRequestedOnlyParametersStructure(obj, parameters_all, parameters_req_names, array_delim) + if nargin < 4 + array_delim = ","; + end + parameters_req = struct(); + for idx=1:length(parameters_req_names) + param_value = parameters_all.(parameters_req_names{idx}); + if (isstring(param_value) && param_value ~= "")... + || (isnumeric(param_value) && isscalar(param_value) && param_value ~= -1) + parameters_req.(parameters_req_names{idx}) = param_value; + elseif isnumeric(param_value) && ~isscalar(param_value) && ~isempty(param_value) + parameters_req.(parameters_req_names{idx}) = strjoin(string(param_value), array_delim); + end + end + end + + function params_string = getParamsString(obj, params, dash_string) + if nargin < 3 + dash_string = "--"; + end + params_fields = fieldnames(params); + params_string = ""; + for idx=1:length(params_fields) + params_string = params_string + " " + dash_string + params_fields{idx} + " " + params.(params_fields{idx}); + end + end + + function command_output = executeCrYOLOCommand(obj, params_string, cryolo_command) + if obj.configuration.use_conda == true + python_run_script_snippet = "LD_LIBRARY_PATH=" + obj.configuration.conda_path + string(filesep) + "lib:$LD_LIBRARY_PATH_COPY"... + + " conda run -n " + obj.configuration.cryolo_env; + end + + cryolo_command_snippet = obj.configuration.conda_path + string(filesep) + "envs"... + + string(filesep) + obj.configuration.cryolo_env + string(filesep) + "bin"... + + string(filesep) + cryolo_command; + python_run_script_snippet = python_run_script_snippet + " python " + cryolo_command_snippet; + command_output = executeCommand(python_run_script_snippet + " " + params_string, false, obj.log_file_id); + end + + end +end + diff --git a/pipeline/GridEnginePipeline.m b/pipeline/GridEnginePipeline.m index 27eff73..cd5d1e6 100644 --- a/pipeline/GridEnginePipeline.m +++ b/pipeline/GridEnginePipeline.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef GridEnginePipeline < Pipeline methods function obj = GridEnginePipeline(configuration_path, default_configuration_path) diff --git a/pipeline/LocalLivePipeline.m b/pipeline/LocalLivePipeline.m index 61d41e9..0638839 100644 --- a/pipeline/LocalLivePipeline.m +++ b/pipeline/LocalLivePipeline.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef LocalLivePipeline < Pipeline methods function obj = LocalLivePipeline(configuration_path, default_configuration_path) diff --git a/pipeline/LocalPipeline.m b/pipeline/LocalPipeline.m index 5a150d8..30cbfdf 100644 --- a/pipeline/LocalPipeline.m +++ b/pipeline/LocalPipeline.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef LocalPipeline < Pipeline methods function obj = LocalPipeline(configuration_path, default_configuration_path) diff --git a/pipeline/Pipeline.m b/pipeline/Pipeline.m index 357a275..3d73c8f 100644 --- a/pipeline/Pipeline.m +++ b/pipeline/Pipeline.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef Pipeline < handle properties diff --git a/pipeline/SlurmPipeline.m b/pipeline/SlurmPipeline.m index b251e30..1609b4a 100644 --- a/pipeline/SlurmPipeline.m +++ b/pipeline/SlurmPipeline.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - classdef SlurmPipeline < Pipeline methods function obj = SlurmPipeline(configuration_path, default_configuration_path) diff --git a/pipeline/runPipeline.m b/pipeline/runPipeline.m index a9db0c2..001773d 100644 --- a/pipeline/runPipeline.m +++ b/pipeline/runPipeline.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function runPipeline(compute_environment, configuration_path, default_configuration_path, starting_tomogram, ending_tomogram, step, gpu) if nargin <= 3 starting_tomogram = -1; diff --git a/pipeline/runTomoBEAR.m b/pipeline/runTomoBEAR.m index 381fd7d..5e12e99 100644 --- a/pipeline/runTomoBEAR.m +++ b/pipeline/runTomoBEAR.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function runTomoBEAR(compute_environment, configuration_path, default_configuration_path, starting_tomogram, ending_tomogram, step, gpu) if nargin <= 2 if fileExists("CONFIGURATION") diff --git a/pipeline/runTomoBear.m b/pipeline/runTomoBear.m index 85691ba..332633d 100644 --- a/pipeline/runTomoBear.m +++ b/pipeline/runTomoBear.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function runTomoBear(compute_environment, configuration_path, default_configuration_path, starting_tomogram, ending_tomogram, step, gpu) if nargin <= 2 if fileExists("CONFIGURATION") diff --git a/startup.m b/startup.m index cf63205..6794ee8 100755 --- a/startup.m +++ b/startup.m @@ -1,18 +1,18 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors -% +% Copyright (c) 2021,2022,2023 TomoBEAR Authors +% % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. -% +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. +% % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. -% -% You should have received a copy of the GNU Affero General Public License +% GNU General Public License for more details. +% +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% diff --git a/utilities/addFileSeparator.m b/utilities/addFileSeparator.m index 8b2148c..38a01d4 100644 --- a/utilities/addFileSeparator.m +++ b/utilities/addFileSeparator.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function path = addFileSeparator(path) if path == "" return; diff --git a/utilities/checkForMultipleExtensions.m b/utilities/checkForMultipleExtensions.m index 78b8ca6..57dcd3d 100644 --- a/utilities/checkForMultipleExtensions.m +++ b/utilities/checkForMultipleExtensions.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function multiple_extensions = checkForMultipleExtensions(files) for i = 1:length(files) [~, ~, extension{i}] = fileparts(files{i}); diff --git a/utilities/cleanCroppedParticlesTable.m b/utilities/cleanCroppedParticlesTable.m index 89d0f74..07192ed 100644 --- a/utilities/cleanCroppedParticlesTable.m +++ b/utilities/cleanCroppedParticlesTable.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function table = cleanCroppedParticlesTable(table, particles_path, as_boxes) % Removes particles from the table which where not cropped (e.g. were % discarded due to exceeding tomogram borders) diff --git a/utilities/combineCCVolumes.m b/utilities/combineCCVolumes.m index d08ff5f..66a21f2 100644 --- a/utilities/combineCCVolumes.m +++ b/utilities/combineCCVolumes.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function [combined_volume, combined_tdrot, combined_tilt, combined_narot] = combineCCVolumes(template_matching_path) dir_list = dir(template_matching_path + "*angle*"); for i = 1:length(dir_list) diff --git a/utilities/combineConfigurations.m b/utilities/combineConfigurations.m index b899d95..4cc83bb 100644 --- a/utilities/combineConfigurations.m +++ b/utilities/combineConfigurations.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function [first_configuration_out, second_configuration_out] = combineConfigurations(first_configurations_in, second_configurations_in) if length(first_configurations_in) > 1 for j = 1:length(first_configurations_in) - 1 diff --git a/utilities/compileTomoBEAR.m b/utilities/compileTomoBEAR.m index 2c96983..c26ee25 100644 --- a/utilities/compileTomoBEAR.m +++ b/utilities/compileTomoBEAR.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function results = compileTomoBEAR(build_destination, default_configuration_path) % TODO: automate compilation %https://www.mathworks.com/help/compiler/compiler.runtime.download.html diff --git a/utilities/copyOrLinkFilesBasedOnSizeThreshold.m b/utilities/copyOrLinkFilesBasedOnSizeThreshold.m index 3478750..890e799 100644 --- a/utilities/copyOrLinkFilesBasedOnSizeThreshold.m +++ b/utilities/copyOrLinkFilesBasedOnSizeThreshold.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function copyOrLinkFilesBasedOnSizeThreshold(source, destination, threshold, log_file_id) file_list = dir(source); file_list(1) = []; diff --git a/utilities/createDynamoLinks.m b/utilities/createDynamoLinks.m index 379fd89..d2fd1a8 100644 --- a/utilities/createDynamoLinks.m +++ b/utilities/createDynamoLinks.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function createDynamoLinks(dynamo_path, link_destination_path) if nargin == 0 error("ERROR: please provide path to dynamo") diff --git a/utilities/createOutputAndScratchFoldersForPipelineStep.m b/utilities/createOutputAndScratchFoldersForPipelineStep.m index 573179f..0df95f9 100644 --- a/utilities/createOutputAndScratchFoldersForPipelineStep.m +++ b/utilities/createOutputAndScratchFoldersForPipelineStep.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function createOutputAndScratchFoldersForPipelineStep(merged_configuration) % TODO: check status if exist(merged_configuration.output_path, "dir")... diff --git a/utilities/createStandardFolder.m b/utilities/createStandardFolder.m index 95aeaf6..f6a92c2 100644 --- a/utilities/createStandardFolder.m +++ b/utilities/createStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function folder = createStandardFolder(configuration, standard_folder, delete) if isfield(configuration, standard_folder) if nargin == 2 diff --git a/utilities/createSymbolicLink.m b/utilities/createSymbolicLink.m index 3ffd4f5..58adab6 100644 --- a/utilities/createSymbolicLink.m +++ b/utilities/createSymbolicLink.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function [destination, output] = createSymbolicLink(source, destination, log_file_id, relative) if nargin == 2 || nargin == 3 relative = true; diff --git a/utilities/createSymbolicLinkInStandardFolder.m b/utilities/createSymbolicLinkInStandardFolder.m index e008539..7d196a8 100644 --- a/utilities/createSymbolicLinkInStandardFolder.m +++ b/utilities/createSymbolicLinkInStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function [output, destination] = createSymbolicLinkInStandardFolder(configuration, source, standard_folder, log_file_id, execute) if nargin <= 4 execute = true; diff --git a/utilities/executeCommand.m b/utilities/executeCommand.m index 38894e3..da45430 100644 --- a/utilities/executeCommand.m +++ b/utilities/executeCommand.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function output = executeCommand(command, ignore_errors, log_file_id, hide_output) if nargin < 4 hide_output = false; diff --git a/utilities/fileExists.m b/utilities/fileExists.m index da4ff3d..ed6cd24 100644 --- a/utilities/fileExists.m +++ b/utilities/fileExists.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function exists = fileExists(fileName) if isfile(fileName) exists = true; diff --git a/utilities/fillSetUpStructIteration.m b/utilities/fillSetUpStructIteration.m index d4e6ca4..bede85d 100644 --- a/utilities/fillSetUpStructIteration.m +++ b/utilities/fillSetUpStructIteration.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function configuration = fillSetUpStructIteration(configuration, j, previous_tomogram_status) configuration.set_up.j = j; configuration.set_up.cumulative_tomogram_status = cumsum(previous_tomogram_status); diff --git a/utilities/fillSetUpStructPipelineStep.m b/utilities/fillSetUpStructPipelineStep.m index fbe7851..3b5e2aa 100644 --- a/utilities/fillSetUpStructPipelineStep.m +++ b/utilities/fillSetUpStructPipelineStep.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function configuration = fillSetUpStructPipelineStep(configuration, i) configuration.set_up = struct; configuration.set_up.i = i; diff --git a/utilities/finishPipelineStep.m b/utilities/finishPipelineStep.m index e85bd30..780b893 100644 --- a/utilities/finishPipelineStep.m +++ b/utilities/finishPipelineStep.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function dynamic_configuration = finishPipelineStep(... dynamic_configuration, pipeline_definition,... configuration_history, i) diff --git a/utilities/generateMaskFromTemplate.m b/utilities/generateMaskFromTemplate.m index 6f9b18b..876000e 100644 --- a/utilities/generateMaskFromTemplate.m +++ b/utilities/generateMaskFromTemplate.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function mask_binarized_smoothed_cleaned = generateMaskFromTemplate(configuration, template) mask = dbandpass(-template, configuration.mask_bandpass); mask_binarized = gather(imbinarize(mask)); diff --git a/utilities/generateParticles.m b/utilities/generateParticles.m index c811680..fc14b7a 100644 --- a/utilities/generateParticles.m +++ b/utilities/generateParticles.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function particles_path = generateParticles(configuration, table_path, binning, box_size, append_tags, force_method) if nargin < 6 diff --git a/utilities/generatePool.m b/utilities/generatePool.m index c23e859..6c4d687 100644 --- a/utilities/generatePool.m +++ b/utilities/generatePool.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function poolobj = generatePool(cores, force, path, debug) persistent previous_cores; if isempty(previous_cores) diff --git a/utilities/generateSetUpStruct.m b/utilities/generateSetUpStruct.m index 59afe17..681407b 100644 --- a/utilities/generateSetUpStruct.m +++ b/utilities/generateSetUpStruct.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function configuration = generateSetUpStruct(configuration, previous_tomogram_status, i ,j) configuration.set_up = struct; configuration.set_up.i = i; diff --git a/utilities/getAlignedEvenTiltStacksFromStandardFolder.m b/utilities/getAlignedEvenTiltStacksFromStandardFolder.m index 4f48109..4a07d8c 100644 --- a/utilities/getAlignedEvenTiltStacksFromStandardFolder.m +++ b/utilities/getAlignedEvenTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getAlignedEvenTiltStacksFromStandardFolder(configuration, flatten) if nargin <= 1 diff --git a/utilities/getAlignedOddTiltStacksFromStandardFolder.m b/utilities/getAlignedOddTiltStacksFromStandardFolder.m index 05c5b67..ce9db1f 100644 --- a/utilities/getAlignedOddTiltStacksFromStandardFolder.m +++ b/utilities/getAlignedOddTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getAlignedOddTiltStacksFromStandardFolder(configuration, flatten) if nargin <= 1 diff --git a/utilities/getAlignedTiltStacksFromStandardFolder.m b/utilities/getAlignedTiltStacksFromStandardFolder.m index 8d3ec80..bfdc182 100644 --- a/utilities/getAlignedTiltStacksFromStandardFolder.m +++ b/utilities/getAlignedTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getAlignedTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getApix.m b/utilities/getApix.m index 57da5ab..e4acbb7 100644 --- a/utilities/getApix.m +++ b/utilities/getApix.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function [rescaled_pixelsize, apix] = getApix(configuration, scaling_binning_factor) if nargin == 1 scaling_binning_factor = 1; diff --git a/utilities/getBinnedAlignedEvenTiltStacksFromStandardFolder.m b/utilities/getBinnedAlignedEvenTiltStacksFromStandardFolder.m index 8da9bcc..217fb23 100644 --- a/utilities/getBinnedAlignedEvenTiltStacksFromStandardFolder.m +++ b/utilities/getBinnedAlignedEvenTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getBinnedAlignedEvenTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getBinnedAlignedOddTiltStacksFromStandardFolder.m b/utilities/getBinnedAlignedOddTiltStacksFromStandardFolder.m index 3c24aa6..dd8649a 100644 --- a/utilities/getBinnedAlignedOddTiltStacksFromStandardFolder.m +++ b/utilities/getBinnedAlignedOddTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getBinnedAlignedOddTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getBinnedAlignedTiltStacksFromStandardFolder.m b/utilities/getBinnedAlignedTiltStacksFromStandardFolder.m index a5a4a84..f02c086 100644 --- a/utilities/getBinnedAlignedTiltStacksFromStandardFolder.m +++ b/utilities/getBinnedAlignedTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getBinnedAlignedTiltStacksFromStandardFolder(configuration, flatten, binning) if nargin <= 2 binning_snippet = "*"; diff --git a/utilities/getBinnedEvenTiltStacksFromStandardFolder.m b/utilities/getBinnedEvenTiltStacksFromStandardFolder.m index d2a313c..7c41248 100644 --- a/utilities/getBinnedEvenTiltStacksFromStandardFolder.m +++ b/utilities/getBinnedEvenTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getBinnedEvenTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getBinnedOddTiltStacksFromStandardFolder.m b/utilities/getBinnedOddTiltStacksFromStandardFolder.m index 0629e1b..cc68b5d 100644 --- a/utilities/getBinnedOddTiltStacksFromStandardFolder.m +++ b/utilities/getBinnedOddTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getBinnedOddTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getBinnedTiltStacksFromStandardFolder.m b/utilities/getBinnedTiltStacksFromStandardFolder.m index 577a2fe..4384ac9 100644 --- a/utilities/getBinnedTiltStacksFromStandardFolder.m +++ b/utilities/getBinnedTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getBinnedTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getBinnedTomogramsFromStandardFolder.m b/utilities/getBinnedTomogramsFromStandardFolder.m index 93f7deb..af3d5b4 100644 --- a/utilities/getBinnedTomogramsFromStandardFolder.m +++ b/utilities/getBinnedTomogramsFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function binned_tomograms = getBinnedTomogramsFromStandardFolder(configuration, flatten, binning) if nargin == 1 flatten = false; diff --git a/utilities/getCpuPoolSize.m b/utilities/getCpuPoolSize.m index 2e2b9bd..fcc2aa0 100644 --- a/utilities/getCpuPoolSize.m +++ b/utilities/getCpuPoolSize.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function poolsize = getCpuPoolSize(cores, cores_requested) % get number of cores available on the used machine diff --git a/utilities/getCtfCorrectedAlignedTiltStacksFromStandardFolder.m b/utilities/getCtfCorrectedAlignedTiltStacksFromStandardFolder.m index 1f049d7..c6fd2a7 100644 --- a/utilities/getCtfCorrectedAlignedTiltStacksFromStandardFolder.m +++ b/utilities/getCtfCorrectedAlignedTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getCtfCorrectedAlignedTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getCtfCorrectedBinnedAlignedEvenTiltStacksFromStandardFolder.m b/utilities/getCtfCorrectedBinnedAlignedEvenTiltStacksFromStandardFolder.m index f04c930..db68696 100644 --- a/utilities/getCtfCorrectedBinnedAlignedEvenTiltStacksFromStandardFolder.m +++ b/utilities/getCtfCorrectedBinnedAlignedEvenTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getCtfCorrectedBinnedAlignedEvenTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getCtfCorrectedBinnedAlignedOddTiltStacksFromStandardFolder.m b/utilities/getCtfCorrectedBinnedAlignedOddTiltStacksFromStandardFolder.m index 9faf172..87ea018 100644 --- a/utilities/getCtfCorrectedBinnedAlignedOddTiltStacksFromStandardFolder.m +++ b/utilities/getCtfCorrectedBinnedAlignedOddTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getCtfCorrectedBinnedAlignedOddTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getCtfCorrectedBinnedAlignedTiltStacksFromStandardFolder.m b/utilities/getCtfCorrectedBinnedAlignedTiltStacksFromStandardFolder.m index 4286259..e651a75 100644 --- a/utilities/getCtfCorrectedBinnedAlignedTiltStacksFromStandardFolder.m +++ b/utilities/getCtfCorrectedBinnedAlignedTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getCtfCorrectedBinnedAlignedTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getCtfCorrectedBinnedEvenTiltStacksFromStandardFolder.m b/utilities/getCtfCorrectedBinnedEvenTiltStacksFromStandardFolder.m index 244e868..b1af272 100644 --- a/utilities/getCtfCorrectedBinnedEvenTiltStacksFromStandardFolder.m +++ b/utilities/getCtfCorrectedBinnedEvenTiltStacksFromStandardFolder.m @@ -1,21 +1,20 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function getCtfCorrectedBinnedEvenTiltStacksFromStandardFolder() end diff --git a/utilities/getCtfCorrectedBinnedEvenTomogramsFromStandardFolder.m b/utilities/getCtfCorrectedBinnedEvenTomogramsFromStandardFolder.m index d32dc1a..4d648b1 100644 --- a/utilities/getCtfCorrectedBinnedEvenTomogramsFromStandardFolder.m +++ b/utilities/getCtfCorrectedBinnedEvenTomogramsFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function binned_tomograms = getCtfCorrectedBinnedEvenTomogramsFromStandardFolder(configuration, flatten, binning) if nargin == 1 flatten = false; diff --git a/utilities/getCtfCorrectedBinnedOddTiltStacksFromStandardFolder.m b/utilities/getCtfCorrectedBinnedOddTiltStacksFromStandardFolder.m index b926bcb..2b0dbbf 100644 --- a/utilities/getCtfCorrectedBinnedOddTiltStacksFromStandardFolder.m +++ b/utilities/getCtfCorrectedBinnedOddTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function getCtfCorrectedBinnedOddTiltStacksFromStandardFolder() end diff --git a/utilities/getCtfCorrectedBinnedOddTomogramsFromStandardFolder.m b/utilities/getCtfCorrectedBinnedOddTomogramsFromStandardFolder.m index f2304ac..56b5ee9 100644 --- a/utilities/getCtfCorrectedBinnedOddTomogramsFromStandardFolder.m +++ b/utilities/getCtfCorrectedBinnedOddTomogramsFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function binned_tomograms = getCtfCorrectedBinnedOddTomogramsFromStandardFolder(configuration, flatten, binning) if nargin == 1 flatten = false; diff --git a/utilities/getCtfCorrectedBinnedTomogramsFromStandardFolder.m b/utilities/getCtfCorrectedBinnedTomogramsFromStandardFolder.m index 798fbde..89aa13b 100644 --- a/utilities/getCtfCorrectedBinnedTomogramsFromStandardFolder.m +++ b/utilities/getCtfCorrectedBinnedTomogramsFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function binned_tomograms = getCtfCorrectedBinnedTomogramsFromStandardFolder(configuration, flatten, binning) if nargin == 1 flatten = false; diff --git a/utilities/getCtfCorrectedTomogramsFromStandardFolder.m b/utilities/getCtfCorrectedTomogramsFromStandardFolder.m index 4bc82d8..e37e92b 100644 --- a/utilities/getCtfCorrectedTomogramsFromStandardFolder.m +++ b/utilities/getCtfCorrectedTomogramsFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tomograms = getCtfCorrectedTomogramsFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getDefocusFiles.m b/utilities/getDefocusFiles.m index 57b05d1..d0fb8f5 100644 --- a/utilities/getDefocusFiles.m +++ b/utilities/getDefocusFiles.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function file_paths = getDefocusFiles(configuration, pattern) % TODO: check for "." in pattern original_mrcs = fieldnames(configuration.tomograms); diff --git a/utilities/getDenoisedCtfCorrectedBinnedTomogramsFromStandardFolder.m b/utilities/getDenoisedCtfCorrectedBinnedTomogramsFromStandardFolder.m index b7ebf54..a97ef34 100644 --- a/utilities/getDenoisedCtfCorrectedBinnedTomogramsFromStandardFolder.m +++ b/utilities/getDenoisedCtfCorrectedBinnedTomogramsFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function binned_tomograms = getDenoisedCtfCorrectedBinnedTomogramsFromStandardFolder(configuration, flatten, binning) if nargin == 1 flatten = false; diff --git a/utilities/getDenoisedCtfCorrectedTomogramsFromStandardFolder.m b/utilities/getDenoisedCtfCorrectedTomogramsFromStandardFolder.m index 6fa855a..86bb654 100644 --- a/utilities/getDenoisedCtfCorrectedTomogramsFromStandardFolder.m +++ b/utilities/getDenoisedCtfCorrectedTomogramsFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function binned_tomograms = getDenoisedCtfCorrectedBinnedTomogramsFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getDoseWeightedSumTiltStacksFromStandardFolder.m b/utilities/getDoseWeightedSumTiltStacksFromStandardFolder.m index 2307910..ef6968b 100644 --- a/utilities/getDoseWeightedSumTiltStacksFromStandardFolder.m +++ b/utilities/getDoseWeightedSumTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getDoseWeightedSumTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getDoseWeightedTiltStacksFromStandardFolder.m b/utilities/getDoseWeightedTiltStacksFromStandardFolder.m index 24ccdb0..af6d6d8 100644 --- a/utilities/getDoseWeightedTiltStacksFromStandardFolder.m +++ b/utilities/getDoseWeightedTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getDoseWeightedTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getDynamoFiles.m b/utilities/getDynamoFiles.m index 3fbae93..bca06a1 100644 --- a/utilities/getDynamoFiles.m +++ b/utilities/getDynamoFiles.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function dynamo_files = getDynamoFiles(prefix) if nargin == 0 prefix = ""; diff --git a/utilities/getEnvironmentProperties.m b/utilities/getEnvironmentProperties.m index aa34a85..d0b70e4 100644 --- a/utilities/getEnvironmentProperties.m +++ b/utilities/getEnvironmentProperties.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function environment_properties = getEnvironmentProperties(configuration) path_to_script = mfilename("fullpath"); path_to_script_split = strsplit(path_to_script, string(filesep)); diff --git a/utilities/getEvenTiltStacksFromStandardFolder.m b/utilities/getEvenTiltStacksFromStandardFolder.m index c16e4bb..af27b7d 100644 --- a/utilities/getEvenTiltStacksFromStandardFolder.m +++ b/utilities/getEvenTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getEvenTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getFidFilePaths.m b/utilities/getFidFilePaths.m index aa209ac..06eb215 100644 --- a/utilities/getFidFilePaths.m +++ b/utilities/getFidFilePaths.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function fiducial_file_paths = getFidFilePaths(configuration) original_mrcs = getOriginalMRCsFromStandardFolder(configuration); batchruntomo_folders = dir(configuration.processing_path + string(filesep)... diff --git a/utilities/getFileCountChanged.m b/utilities/getFileCountChanged.m index caac2a4..18fa367 100644 --- a/utilities/getFileCountChanged.m +++ b/utilities/getFileCountChanged.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function file_count_changed = getFileCountChanged(configuration) [file_list, ~] = getOriginalMRCsorTIFs(configuration); if ~isfield(configuration, "file_count")... diff --git a/utilities/getFilePathsFromLastBatchruntomoRun.m b/utilities/getFilePathsFromLastBatchruntomoRun.m index b655cfc..abe8de1 100644 --- a/utilities/getFilePathsFromLastBatchruntomoRun.m +++ b/utilities/getFilePathsFromLastBatchruntomoRun.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function file_paths = getFilePathsFromLastBatchruntomoRun(configuration, file_extension) batchruntomo_folders = dir(configuration.processing_path + string(filesep)... + configuration.output_folder + string(filesep) + "*_BatchRunTomo_*"); diff --git a/utilities/getFilesFromLastModuleRun.m b/utilities/getFilesFromLastModuleRun.m index 1572623..f7c471a 100644 --- a/utilities/getFilesFromLastModuleRun.m +++ b/utilities/getFilesFromLastModuleRun.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function file_paths = getFilesFromLastModuleRun(configuration, module_name, file_extension, choice) if nargin == 3 choice = "last"; diff --git a/utilities/getFilesWithMatchingPatternFromLastBatchruntomoRun.m b/utilities/getFilesWithMatchingPatternFromLastBatchruntomoRun.m index 14756e0..de79e36 100644 --- a/utilities/getFilesWithMatchingPatternFromLastBatchruntomoRun.m +++ b/utilities/getFilesWithMatchingPatternFromLastBatchruntomoRun.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function file_paths = getFilesWithMatchingPatternFromLastBatchruntomoRun(configuration, pattern) % TODO: check for "." in pattern original_mrcs = fieldnames(configuration.tomograms); diff --git a/utilities/getHeightAndWidthFromHeader.m b/utilities/getHeightAndWidthFromHeader.m index 2e3b479..ce88c34 100644 --- a/utilities/getHeightAndWidthFromHeader.m +++ b/utilities/getHeightAndWidthFromHeader.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function [width, height, z] = getHeightAndWidthFromHeader(file_path, log_file_id) if iscell(file_path) command = sprintf("header %s", file_path{1}); diff --git a/utilities/getIMODVersion.m b/utilities/getIMODVersion.m index 6b651d7..834d0ed 100644 --- a/utilities/getIMODVersion.m +++ b/utilities/getIMODVersion.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function imod_version = getIMODVersion() output = executeCommand("3dmod -h", true, -1, true); output_lines = textscan(output,"%s","Delimiter","","endofline","\n"); diff --git a/utilities/getMask.m b/utilities/getMask.m index 4e8a03d..4aa2ad2 100644 --- a/utilities/getMask.m +++ b/utilities/getMask.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function mask = getMask(configuration, path) file_path = configuration.processing_path + string(filesep)... + configuration.output_folder + string(filesep)... diff --git a/utilities/getMetaDataFilePath.m b/utilities/getMetaDataFilePath.m index 3e6da62..f95c009 100644 --- a/utilities/getMetaDataFilePath.m +++ b/utilities/getMetaDataFilePath.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function meta_data_file_path = getMetaDataFilePath(configuration) meta_data_folder_path = getMetaDataFolderPath(configuration); meta_data_file_path = meta_data_folder_path + string(filesep)... diff --git a/utilities/getMetaDataFolderPath.m b/utilities/getMetaDataFolderPath.m index b8a2e58..9156250 100644 --- a/utilities/getMetaDataFolderPath.m +++ b/utilities/getMetaDataFolderPath.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function meta_data_folder_path = getMetaDataFolderPath(configuration) meta_data_folder_path = configuration.processing_path + string(filesep) + configuration.output_folder + string(filesep) + configuration.meta_data_folder; end diff --git a/utilities/getMinAndMaxTiltAnglesFromTiltFile.m b/utilities/getMinAndMaxTiltAnglesFromTiltFile.m index f247800..8e6b9bd 100644 --- a/utilities/getMinAndMaxTiltAnglesFromTiltFile.m +++ b/utilities/getMinAndMaxTiltAnglesFromTiltFile.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function min_and_max_tilt_angles = getMinAndMaxTiltAnglesFromTiltFile(configuration) tomograms = getTomograms(configuration, true); batchruntomo_folders = dir(configuration.processing_path + string(filesep)... diff --git a/utilities/getMotionCorrectedMRCsFromStandardFolder.m b/utilities/getMotionCorrectedMRCsFromStandardFolder.m index 3e4dfe0..72cef84 100644 --- a/utilities/getMotionCorrectedMRCsFromStandardFolder.m +++ b/utilities/getMotionCorrectedMRCsFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function motion_corrected_mrcs = getMotionCorrectedMRCsFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getOddTiltStacksFromStandardFolder.m b/utilities/getOddTiltStacksFromStandardFolder.m index c04b303..bba2751 100644 --- a/utilities/getOddTiltStacksFromStandardFolder.m +++ b/utilities/getOddTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getOddTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getOriginalEERs.m b/utilities/getOriginalEERs.m index 5490195..066fced 100644 --- a/utilities/getOriginalEERs.m +++ b/utilities/getOriginalEERs.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function [original_eers, tif_flag] = getOriginalEERs(configuration) tif_flag = 0; if isfield(configuration, "tomogram_input_prefix")... diff --git a/utilities/getOriginalFiles.m b/utilities/getOriginalFiles.m index 7515b2b..f7157f4 100644 --- a/utilities/getOriginalFiles.m +++ b/utilities/getOriginalFiles.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function original_files = getOriginalFiles(configuration, grouped) if nargin == 1 grouped = false; diff --git a/utilities/getOriginalFilesFromFilePaths.m b/utilities/getOriginalFilesFromFilePaths.m index 6b3185f..79d8e72 100644 --- a/utilities/getOriginalFilesFromFilePaths.m +++ b/utilities/getOriginalFilesFromFilePaths.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function original_files = getOriginalFilesFromFilePaths(file_paths, grouped) if nargin == 1 grouped = false; diff --git a/utilities/getOriginalMRCs.m b/utilities/getOriginalMRCs.m index ec2472f..a847fcb 100644 --- a/utilities/getOriginalMRCs.m +++ b/utilities/getOriginalMRCs.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function original_mrcs = getOriginalMRCs(configuration) if isfield(configuration, "tomogram_input_prefix")... && configuration.tomogram_input_prefix ~= "" diff --git a/utilities/getOriginalMRCsFromStandardFolder.m b/utilities/getOriginalMRCsFromStandardFolder.m index 09029fe..f5cde59 100644 --- a/utilities/getOriginalMRCsFromStandardFolder.m +++ b/utilities/getOriginalMRCsFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function original_mrcs = getOriginalMRCsFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getOriginalMRCsorTIFs.m b/utilities/getOriginalMRCsorTIFs.m index 63340d7..f9acb64 100644 --- a/utilities/getOriginalMRCsorTIFs.m +++ b/utilities/getOriginalMRCsorTIFs.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function [original_files, tif_flag] = getOriginalMRCsorTIFs(configuration, grouped) if nargin < 2 grouped = false; diff --git a/utilities/getOriginalTIFs.m b/utilities/getOriginalTIFs.m index 331ec3b..6b83449 100644 --- a/utilities/getOriginalTIFs.m +++ b/utilities/getOriginalTIFs.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function [original_tifs, tif_flag] = getOriginalTIFs(configuration) tif_flag = 1; if isfield(configuration, "tomogram_input_prefix")... diff --git a/utilities/getOriginalTIFsFromStandardFolder.m b/utilities/getOriginalTIFsFromStandardFolder.m index 0537b38..5d5b50a 100644 --- a/utilities/getOriginalTIFsFromStandardFolder.m +++ b/utilities/getOriginalTIFsFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function original_tifs = getOriginalTIFsFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getPixelSizeFromHeader.m b/utilities/getPixelSizeFromHeader.m index 489413a..0a78dd5 100644 --- a/utilities/getPixelSizeFromHeader.m +++ b/utilities/getPixelSizeFromHeader.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function apix = getPixelSizeFromHeader(file_path, log_file_id) if iscell(file_path) command = sprintf("header %s", file_path{1}); diff --git a/utilities/getTemplate.m b/utilities/getTemplate.m index d518cab..4e0c334 100644 --- a/utilities/getTemplate.m +++ b/utilities/getTemplate.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function template = getTemplate(configuration, path) file_path = configuration.processing_path + string(filesep)... + configuration.output_folder + string(filesep)... diff --git a/utilities/getTiltAngles.m b/utilities/getTiltAngles.m index d775eee..3f942f8 100644 --- a/utilities/getTiltAngles.m +++ b/utilities/getTiltAngles.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_angles = getTiltAngles(configuration, get_all_angles) if nargin == 1 get_all_angles = false; diff --git a/utilities/getTiltIndexAngleMapping.m b/utilities/getTiltIndexAngleMapping.m index 5202249..94c2893 100644 --- a/utilities/getTiltIndexAngleMapping.m +++ b/utilities/getTiltIndexAngleMapping.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_index_angle_mapping = getTiltIndexAngleMapping(configuration) meta_data_path = configuration.processing_path + string(filesep)... + configuration.output_folder + string(filesep)... diff --git a/utilities/getTiltStacks.m b/utilities/getTiltStacks.m index c464251..03a5ab1 100644 --- a/utilities/getTiltStacks.m +++ b/utilities/getTiltStacks.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getTiltStacks(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getTiltStacksFromPreviousPipelineStepFolders.m b/utilities/getTiltStacksFromPreviousPipelineStepFolders.m index 3d960bb..bba6658 100644 --- a/utilities/getTiltStacksFromPreviousPipelineStepFolders.m +++ b/utilities/getTiltStacksFromPreviousPipelineStepFolders.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getTiltStacksFromPreviousPipelineStepFolders(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getTiltStacksFromStandardFolder.m b/utilities/getTiltStacksFromStandardFolder.m index 00097ca..77bfd5d 100644 --- a/utilities/getTiltStacksFromStandardFolder.m +++ b/utilities/getTiltStacksFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tilt_stacks = getTiltStacksFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getTltxfFilePaths.m b/utilities/getTltxfFilePaths.m index ca7a6d0..7cc600e 100644 --- a/utilities/getTltxfFilePaths.m +++ b/utilities/getTltxfFilePaths.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tiltxf_file_paths = getTltxfFilePaths(configuration) original_mrcs = getOriginalMRCsFromStandardFolder(configuration); batchruntomo_folders = dir(configuration.processing_path + string(filesep)... diff --git a/utilities/getTomograms.m b/utilities/getTomograms.m index 6737a69..fd31c26 100644 --- a/utilities/getTomograms.m +++ b/utilities/getTomograms.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tomograms = getTomograms(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getTomogramsFromPreviousPipelineStepFolders.m b/utilities/getTomogramsFromPreviousPipelineStepFolders.m index aab3565..86693d6 100644 --- a/utilities/getTomogramsFromPreviousPipelineStepFolders.m +++ b/utilities/getTomogramsFromPreviousPipelineStepFolders.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tomograms = getTomogramsFromPreviousPipelineStepFolders(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getTomogramsFromStandardFolder.m b/utilities/getTomogramsFromStandardFolder.m index f2d0951..19b4b2f 100644 --- a/utilities/getTomogramsFromStandardFolder.m +++ b/utilities/getTomogramsFromStandardFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function tomograms = getTomogramsFromStandardFolder(configuration, flatten) if nargin == 1 flatten = false; diff --git a/utilities/getUnprocessedTomograms.m b/utilities/getUnprocessedTomograms.m index d8e3fea..4ca9401 100644 --- a/utilities/getUnprocessedTomograms.m +++ b/utilities/getUnprocessedTomograms.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function [dynamic_configuration, file_count_changed] = getUnprocessedTomograms(configuration, log_file_id) dynamic_configuration = struct; original_files = getOriginalFiles(configuration, false); diff --git a/utilities/getXfOrAlnFilePaths.m b/utilities/getXfOrAlnFilePaths.m index c942d02..9d45650 100644 --- a/utilities/getXfOrAlnFilePaths.m +++ b/utilities/getXfOrAlnFilePaths.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function xf_file_path = getXfOrAlnFilePaths(configuration, output_path, tomogram_name) xf_file_path = getFilesFromLastModuleRun(configuration,"AreTomo","xf","last"); if ~isempty(xf_file_path) diff --git a/utilities/gpuEllipsoid.m b/utilities/gpuEllipsoid.m index d55eb66..d1d73ef 100644 --- a/utilities/gpuEllipsoid.m +++ b/utilities/gpuEllipsoid.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function vol = gpuEllipsoid(radii,siz,position,smoothing_pixels) if ndims(siz) > 2 siz = size(siz); diff --git a/utilities/gpuEulerToMatrix.m b/utilities/gpuEulerToMatrix.m index cfa57c9..a171f79 100644 --- a/utilities/gpuEulerToMatrix.m +++ b/utilities/gpuEulerToMatrix.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function transformation_matrices = gpuEulerToMatrix(angles, varargin) defaultMatrixOperation = "crotation"; diff --git a/utilities/histogramEqualization.m b/utilities/histogramEqualization.m index 4845a42..233c53e 100644 --- a/utilities/histogramEqualization.m +++ b/utilities/histogramEqualization.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - % NOTE: https://stackoverflow.com/questions/26763974/histogram-matching-of-two-images-without-using-histeq function output_image = histogramEqualization(image_1, image_2) bin_counts = 2^16; diff --git a/utilities/increaseFolderNumber.m b/utilities/increaseFolderNumber.m index 5312472..59399ca 100644 --- a/utilities/increaseFolderNumber.m +++ b/utilities/increaseFolderNumber.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function [folder, previous_folder] = increaseFolderNumber(configuration, folder) dir_list = dir(folder + "*"); if ~isempty(dir_list) && configuration.increase_folder_numbers == true diff --git a/utilities/iteration.m b/utilities/iteration.m index 3997a3e..6e538d6 100644 --- a/utilities/iteration.m +++ b/utilities/iteration.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function [dynamic_configuration_out, status] = iteration(merged_configuration, pipeline_definition, tomogram_name, previous_tomogram_status) dynamic_configuration_out = struct; if isfield(merged_configuration, "skip") && merged_configuration.skip == true diff --git a/utilities/loadJSON.m b/utilities/loadJSON.m index d131871..f2791c2 100644 --- a/utilities/loadJSON.m +++ b/utilities/loadJSON.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function configuration = loadJSON(file_path) fid = fopen(file_path); text = fscanf(fid, "%s"); diff --git a/utilities/makeEvenVolumeDimensions.m b/utilities/makeEvenVolumeDimensions.m index 84c53a6..9d62d9e 100644 --- a/utilities/makeEvenVolumeDimensions.m +++ b/utilities/makeEvenVolumeDimensions.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function volume_out = makeEvenVolumeDimensions(volume_in, mean_value) if mod(length(volume_in), 2) == 1 % TODO: adjust for non cubic sizes diff --git a/utilities/mergeConfigurations.m b/utilities/mergeConfigurations.m index 250e04f..0e11fdf 100644 --- a/utilities/mergeConfigurations.m +++ b/utilities/mergeConfigurations.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function merged_configurations = mergeConfigurations(first_configuration, second_configuration, pipeline_step, script_name) printVariable(first_configuration); printVariable(second_configuration); diff --git a/utilities/ownDbox.m b/utilities/ownDbox.m index 6e76b73..1f82f55 100644 --- a/utilities/ownDbox.m +++ b/utilities/ownDbox.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function ownDbox(source_folder, destination_folder, batch, delete_source_folder, padding) if nargin < 2 error("ERROR: to create dbox folder source and destination is needed!"); diff --git a/utilities/printToFile.m b/utilities/printToFile.m index cc993a8..054ce5c 100644 --- a/utilities/printToFile.m +++ b/utilities/printToFile.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function printToFile(file_id, message) fprintf(file_id, "%s\n", message); end diff --git a/utilities/printVariable.m b/utilities/printVariable.m index f796dbd..e516ea2 100644 --- a/utilities/printVariable.m +++ b/utilities/printVariable.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - % NOTE: works only for named variables function printVariable(variable, debug) if nargin == 1 diff --git a/utilities/printVariableToString.m b/utilities/printVariableToString.m index 262ce03..e2639dc 100644 --- a/utilities/printVariableToString.m +++ b/utilities/printVariableToString.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - % NOTE: works only for named variables function variable_string = printVariableToString(variable) if iscell(variable) diff --git a/utilities/readConfigurationsFromFolder.m b/utilities/readConfigurationsFromFolder.m index c443932..d0aeb41 100644 --- a/utilities/readConfigurationsFromFolder.m +++ b/utilities/readConfigurationsFromFolder.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function configurations = readConfigurationsFromFolder(folder) configuration_file_path = dir(folder + string(filesep) + "output.json"); if isempty(configuration_file_path) diff --git a/utilities/removeDynamoLinks.m b/utilities/removeDynamoLinks.m index 339239e..9cfe7cc 100644 --- a/utilities/removeDynamoLinks.m +++ b/utilities/removeDynamoLinks.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function removeDynamoLinks() linked_files = getDynamoFiles("dynamo"); diff --git a/utilities/saveJSON.m b/utilities/saveJSON.m index 13cfbef..ea432f2 100644 --- a/utilities/saveJSON.m +++ b/utilities/saveJSON.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function saveJSON(path, variable) fid_json = fopen(path, "wt" ); variable_json = jsonencode(variable, "ConvertInfAndNaN", false); diff --git a/utilities/sortDirOutputByPipelineStepNumbering.m b/utilities/sortDirOutputByPipelineStepNumbering.m index 70aaed9..da06d5d 100644 --- a/utilities/sortDirOutputByPipelineStepNumbering.m +++ b/utilities/sortDirOutputByPipelineStepNumbering.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function order = sortDirOutputByPipelineStepNumbering(dir_list, configuration) for j = 1:length(dir_list) dir_list_real_indices{j} = strsplit(dir_list(j).name, "_"); diff --git a/utilities/template_matching_post_processing_iteration.m b/utilities/template_matching_post_processing_iteration.m index 5312731..aa3551d 100644 --- a/utilities/template_matching_post_processing_iteration.m +++ b/utilities/template_matching_post_processing_iteration.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function [tab_tomo, temporary_files] = template_matching_post_processing_iteration(configuration, binned_tomograms_paths_filtered, output_path, mask_erase) [folder, name, extension] = fileparts(binned_tomograms_paths_filtered.folder); name_splitted = strsplit(name, "_"); diff --git a/utilities/versionGreaterThan.m b/utilities/versionGreaterThan.m index 91400e0..8e5b591 100644 --- a/utilities/versionGreaterThan.m +++ b/utilities/versionGreaterThan.m @@ -1,22 +1,21 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % This file is part of the TomoBEAR software. -% Copyright (c) 2021-2023 TomoBEAR Authors +% Copyright (c) 2021,2022,2023 TomoBEAR Authors % % This program is free software: you can redistribute it and/or modify -% it under the terms of the GNU Affero General Public License as -% published by the Free Software Foundation, either version 3 of the -% License, or (at your option) any later version. +% it under the terms of the GNU General Public License as published +% by the Free Software Foundation, either version 3 of the License, +% or (at your option) any later version. % % This program is distributed in the hope that it will be useful, % but WITHOUT ANY WARRANTY; without even the implied warranty % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -% GNU Affero General Public License for more details. +% GNU General Public License for more details. % -% You should have received a copy of the GNU Affero General Public License +% You should have received a copy of the GNU General Public License % along with this program. If not, see . %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - function result = versionGreaterThan(version, version_to_be_compared_to, delimiter) if nargin == 2 delimiter = "."; diff --git a/wiki/Additional-Software-Citation.md b/wiki/Additional-Software-Citation.md deleted file mode 100644 index 576ebde..0000000 --- a/wiki/Additional-Software-Citation.md +++ /dev/null @@ -1,62 +0,0 @@ -# Additional Software Citations - -As ```TomoBEAR``` wraps different third-party software packages for different processing stages, these need to be properly cited, once used under ```TomoBEAR```. Please, note that by using these software packages you are assumed to read the corresponding license files and fully accept them. - -#### IMOD - -General usage: -- Mastronarde DN & Held SR (2017) Automated tilt series alignment and tomographic -reconstruction in IMOD. *J Struct Biol* 197: 102–113. DOI: [10.1016/j.jsb.2016.07.011](https://www.sciencedirect.com/science/article/pii/S1047847716301526?via%3Dihub) - -Non-linear Anisotropic Diffusion filter: -- Frangakis A & Hegerl R (2001) Noise reduction in electron tomographic reconstructions using nonlinear anisotropic diffusion. *J Struct Biol* 135: 239-205. DOI: [10.1006/jsbi.2001.4406](https://www.sciencedirect.com/science/article/pii/S1047847701944065?via%3Dihub). - -#### Dynamo - -Main reference for its usage under TomoBEAR: -- Castaño-Díez D, Kudryashev M, Arheit M & Stahlberg H (2012) Dynamo: A flexible, user-friendly development tool for subtomogram averaging of cryo-EM data in high-performance computing environments. *J Struct Biol* 186:139-151. DOI: [10.1016/j.jsb.2011.12.017](https://www.sciencedirect.com/science/article/pii/S1047847711003650?via%3Dihub) - -If you pick particles manually using Dynamo Catalogue as well, please cite also the following publication: -- Castaño-Díez D, Kudryashev M & Stahlberg H (2016) Dynamo Catalogue: Geometrical tools -and data management for particle picking in subtomogram averaging of cryo-electron -tomograms. *J Struct Biol* 197(2):135-144. DOI: [10.1016/j.jsb.2016.06.005](https://www.sciencedirect.com/science/article/pii/S1047847716301113?via%3Dihub) - -#### MotionCor2 - -- Zheng SQ, Palovcak E, Armache J-P, Verba KA, Cheng Y & Agard DA (2017) MotionCor2: -anisotropic correction of beam-induced motion for improved cryo-electron microscopy. *Nat Methods* 14: 331–332. DOI: [10.1038/nmeth.4193](https://www.nature.com/articles/nmeth.4193) - -#### AreTomo - -- Zheng S, Wolff G, Greenan G, Chen Z, Faas FGA, Bárcena M, Koster AJ, Cheng Y & Agard DA -(2022) AreTomo: An integrated software package for automated marker-free, -motion-corrected cryo-electron tomographic alignment and reconstruction. *J Struct Biol X* -6: 100068. DOI: [10.1016/j.yjsbx.2022.100068](https://www.sciencedirect.com/science/article/pii/S2590152422000095?via%3Dihub) - -#### Gctf - -For all three GCTF versions usage (v1.06/v1.18/Gautomatch cu10.1) please cite the following publication: -- Zhang K (2016) Gctf: Real-time CTF determination and correction. *J Struct Biol* 193: 1–12. DOI: [10.1016/j.jsb.2015.11.003](https://www.sciencedirect.com/science/article/pii/S1047847715301003?via%3Dihub) - -#### CTFFIND4 - -CTFFIND4: -- Rohou A & Grigorieff N (2015) CTFFIND4: Fast and accurate defocus estimation from electron micrographs. *J Struct Biol* 192(2): 216-221. DOI: [10.1016/j.jsb.2015.08.008](https://www.sciencedirect.com/science/article/pii/S1047847715300460?via%3Dihub) - -CTFFIND4 from cisTEM package: -- Grant T, Rohou A & Grigorieff N (2018) cisTEM, user-friendly software for single-particle image processing. *Elife* 7: e35383. DOI: [10.7554/eLife.35383](https://elifesciences.org/articles/35383) - -#### IsoNet - -- Liu YT, Zhang H, Wang H et al. (2022) Isotropic reconstruction for electron tomography with deep learning. *Nat Commun* 13: 6482. DOI: [10.1038/s41467-022-33957-8](https://www.nature.com/articles/s41467-022-33957-8) - -#### CryoCARE - -- Buchholz TO, Jordan M, Pigino G & Jug F (2018) Cryo-CARE: Content-Aware Image Restoration for Cryo-Transmission Electron Microscopy Data. *arXiv* arXiv:1810.05420. DOI: [10.48550/arXiv.1810.05420](https://arxiv.org/abs/1810.05420) - - -#### SUSAN - -If you used SUSAN, please, cite as the following: - -- Sánchez RM (2023) rkms86/SUSAN: Release for TomoBEAR (v0.1-RC1-TomoBEAR). *Zenodo*. DOI: [10.5281/zenodo.7950904](https://doi.org/10.5281/zenodo.7950904) diff --git a/wiki/External-Software.md b/wiki/External-Software.md new file mode 100644 index 0000000..ca1891e --- /dev/null +++ b/wiki/External-Software.md @@ -0,0 +1,157 @@ +# External Software + +As ```TomoBEAR``` wraps different third-party software packages for different processing stages, here we provide the corresponding installation and citation notes. + +Please remember that along with the ```TomoBEAR``` you **must cite all the used external packages as well**. Please, note as well that **by using these software packages it is assumed that you have read the corresponding license files and fully accept them**. + +TomoBEAR modules dependencies on the external software are available on the [Modules page](https://github.com/KudryashevLab/TomoBEAR/wiki/Modules). + +## Contents + +- [Mandatory software](#mandatory-software) +- [Optional software](#optional-software) +- [Optional Python-based software](#optional-python-based-software) + +## Mandatory software + +### IMOD + +**Installation** + +Get the IMOD of version `4.11.24` from the [IMOD download page](https://bio3d.colorado.edu/imod/download.html). +As well, we have tested TomoBEAR under `IMOD-4.9.12` which you may find on the [IMOD package archive page](https://bio3d.colorado.edu/imod/download.html#Archive). + +**Citation** + +General usage: +- Mastronarde DN & Held SR (2017) Automated tilt series alignment and tomographic +reconstruction in IMOD. *J Struct Biol* 197: 102–113. DOI: [10.1016/j.jsb.2016.07.011](https://www.sciencedirect.com/science/article/pii/S1047847716301526?via%3Dihub) + +Non-linear Anisotropic Diffusion filter: +- Frangakis A & Hegerl R (2001) Noise reduction in electron tomographic reconstructions using nonlinear anisotropic diffusion. *J Struct Biol* 135: 239-205. DOI: [10.1006/jsbi.2001.4406](https://www.sciencedirect.com/science/article/pii/S1047847701944065?via%3Dihub). + +### Dynamo + +**Installation** + +For the non standalone version of TomoBEAR you need a Dynamo version with tilt stack alignment capabilities. The newest version can be [downloaded from here](https://wiki.dynamo.biozentrum.unibas.ch/w/index.php/Downloads). TomoBEAR was tested using the Dynamo of version `1.1.532` with `MCR-9.9.0` for `GLNXA64` system and GPU-dependent routines were compiled under `CUDA-11.5`. + +To minimize the dependencies on different CUDA versions it is advised to recompile the CUDA kernel for averaging with the newest CUDA version which is at best already available on your machine. If not please revise the chapter CUDA on this page. + +To recompile the kernel you just need to the location where dynamo was extracted and access the folder ```cuda``` inside. There you will find a file called ```makefile``` which you need to open and modify the second line containing the variable ```CUDA_ROOT=```. Please put in there the path to your most recent CUDA release available on the system. + +To recompile just execute the following two commands: + +* `make clean` +* `make all` + +**Citation** + +Main reference for its usage under TomoBEAR: +- Castaño-Díez D, Kudryashev M, Arheit M & Stahlberg H (2012) Dynamo: A flexible, user-friendly development tool for subtomogram averaging of cryo-EM data in high-performance computing environments. *J Struct Biol* 186:139-151. DOI: [10.1016/j.jsb.2011.12.017](https://www.sciencedirect.com/science/article/pii/S1047847711003650?via%3Dihub) + +If you pick particles manually using Dynamo Catalogue as well, please cite also the following publication: +- Castaño-Díez D, Kudryashev M & Stahlberg H (2016) Dynamo Catalogue: Geometrical tools +and data management for particle picking in subtomogram averaging of cryo-electron +tomograms. *J Struct Biol* 197(2):135-144. DOI: [10.1016/j.jsb.2016.06.005](https://www.sciencedirect.com/science/article/pii/S1047847716301113?via%3Dihub) + +## Optional software + +### MotionCor2 + +**Installation** + +Head to the +[MotionCor2](https://docs.google.com/forms/d/e/1FAIpQLSfAQm5MA81qTx90W9JL6ClzSrM77tytsvyyHh1ZZWrFByhmfQ/viewform) +download page. There you need to register and download MotionCor2. A MotionCor2 version greater than `1.4.0` is desired. [Alternative download link](https://emcore.ucsf.edu/ucsf-software). + +**Citation** + +- Zheng SQ, Palovcak E, Armache J-P, Verba KA, Cheng Y & Agard DA (2017) MotionCor2: +anisotropic correction of beam-induced motion for improved cryo-electron microscopy. *Nat Methods* 14: 331–332. DOI: [10.1038/nmeth.4193](https://www.nature.com/articles/nmeth.4193) + +### AreTomo + +**Installation** + +Head to the [AreTomo download page](https://drive.google.com/drive/folders/1Z7pKVEdgMoNaUmd_cOFhlt-QCcfcwF3_). There you can find different AreTomo versions along with the documentation. TomoBEAR was tested using AreTomo of version `1.3.3` for `CUDA-11.5` (release date mark: 11212022). + +**Citation** + +- Zheng S, Wolff G, Greenan G, Chen Z, Faas FGA, Bárcena M, Koster AJ, Cheng Y & Agard DA +(2022) AreTomo: An integrated software package for automated marker-free, +motion-corrected cryo-electron tomographic alignment and reconstruction. *J Struct Biol X* +6: 100068. DOI: [10.1016/j.yjsbx.2022.100068](https://www.sciencedirect.com/science/article/pii/S2590152422000095?via%3Dihub) + +### Gctf + +**Installation** + +You can download and try one of the following GCTF versions: +- [Gctf v1.06](https://www2.mrc-lmb.cam.ac.uk/download/gctf_v1-06-and-examples/) - the version tested to be working under TomoBEAR; +- [Gctf v1.18](https://www2.mrc-lmb.cam.ac.uk/download/special-version-for-phase-plate-gctf_v1-18/) - special version for phase plate data, as well tested to be working under TomoBEAR; +- [Gctf Gautomatch cu10.1](https://www2.mrc-lmb.cam.ac.uk/download/gctf_gautomatch_cu10-1-tar-gz/) - version for CUDA-10 (should work under TomoBEAR, but was not tested). + +**Citation** + +For all three GCTF versions usage (v1.06/v1.18/Gautomatch cu10.1) please cite the following publication: +- Zhang K (2016) Gctf: Real-time CTF determination and correction. *J Struct Biol* 193: 1–12. DOI: [10.1016/j.jsb.2015.11.003](https://www.sciencedirect.com/science/article/pii/S1047847715301003?via%3Dihub) + +### CTFFIND4 + +**Installation** + +Head to the [CTFFIND4](https://grigoriefflab.umassmed.edu/ctf_estimation_ctffind_ctftilt) or [cisTEM](https://cistem.org/) download page. There you can find CTFFIND4/cisTEM source code and/or binaries. TomoBEAR was tested using CTFIND4 of `4.1.14` version released on May 8 2020. + +> **Note** +>
In the following releases we are also planning to include CTFFIND4 updated version with tilted images support from the development version of the cisTEM package (https://cistem.org/development). + +**Citation** + +CTFFIND4: +- Rohou A & Grigorieff N (2015) CTFFIND4: Fast and accurate defocus estimation from electron micrographs. *J Struct Biol* 192(2): 216-221. DOI: [10.1016/j.jsb.2015.08.008](https://www.sciencedirect.com/science/article/pii/S1047847715300460?via%3Dihub) + +CTFFIND4 from cisTEM package: +- Grant T, Rohou A & Grigorieff N (2018) cisTEM, user-friendly software for single-particle image processing. *Elife* 7: e35383. DOI: [10.7554/eLife.35383](https://elifesciences.org/articles/35383) + +### SUSAN + +**Installation** + +To install and use SUSAN follow the instructions in the [SUSAN](https://github.com/rkms86/SUSAN) code repository. TomoBEAR was tested with the SUSAN special release for TomoBEAR `v0.1-RC1-TomoBEAR`, posted on Zenodo (see citation link below). However, SUSAN uses rolling updates CI/CD model (as of July 2023) and hence no releases are currently issued, so we would encourage you to get the newer SUSAN version and try it with TomoBEAR, reporting us about any problems you faced with either TomoBEAR or SUSAN tools. + +**Citation** + +- Sánchez RM (2023) rkms86/SUSAN: Release for TomoBEAR (v0.1-RC1-TomoBEAR). *Zenodo*. DOI: [10.5281/zenodo.7950904](https://doi.org/10.5281/zenodo.7950904) + +## Optional Python-based software + +### Anaconda + +TomoBEAR can use various python based techniques to extend its functionality like using a neural net-based picker (crYOLO), denoising (cryoCARE) or missing wedge reconstruction algorithm (IsoNet). For that, it assumes you have Anaconda or Miniconda installed. For that either use your OS-included package manager or install it from the [Anaconda web page](https://www.anaconda.com/products/individual). You can also take the miniconda installation to save on space and inodes. + +### IsoNet + +**Installation** + +IsoNet is a DL framework based on convolutional neural nets (CNNs) and the U-net architecture which can be trained to both denoise and reconstruct missing wedge on cryo-elecrtron microscopy data. In order to use IsoNet under TomoBEAR, please clone the [IsoNet source code](https://github.com/IsoNet-cryoET/IsoNet) and follow the instructions on that page to setup the corresponding Python environment. TomoBEAR was tested using IsoNet of version `0.2` along with `CUDA-11.5` and the following versions of the main IsoNet dependencies: `python`=3.9, `cudatoolkit`=11.5, `cudnn`=8.3, `tensorflow-gpu`=2.11. + +The original IsoNet version has limited missing wedge angular range of -60...+60, however there is an extended IsoNet version which allows for arbitrary parametrized missing wedge angular range on the ```mw_angle``` branch of the IsoNet original repository. TomoBEAR supports this version as well, so if you want to use it, you need to clone this branch by: +```bash +git clone --branch mw_angle https://github.com/IsoNet-cryoET/IsoNet.git IsoNet_mw +``` +where ```IsoNet_mw``` is the folder name where this IsoNet version will be cloned so that original IsoNet version is not overwritten if you already got one. + +**Citation** + +- Liu YT, Zhang H, Wang H et al. (2022) Isotropic reconstruction for electron tomography with deep learning. *Nat Commun* 13: 6482. DOI: [10.1038/s41467-022-33957-8](https://www.nature.com/articles/s41467-022-33957-8) + +### crYOLO + +**Installation** + +[crYOLO](https://cryolo.readthedocs.io/en/stable/index.html) is a DL framework based on convolutional neural nets (CNNs) which utilizes the popular **You Only Look Once (YOLO)** object detection system. crYOLO can be trained to pick particles XYZ positions from cryo-elecrtron microscopy data. If you want to try crYOLO under TomoBEAR, please follow the [crYOLO installation instructions](https://cryolo.readthedocs.io/en/stable/installation.html) to setup the corresponding Python (conda) environment. TomoBEAR was tested using crYOLO of version `1.9.3` along with `CUDA-11.5` and the following versions of the main crYOLO dependencies: `python`=3.8, `nvidia-cudnn-cu115`=8.3, `nvidia-tensorflow`=1.15. + +**Citation** + +- Wagner T, Merino F, Stabrin M et al. (2019). SPHIRE-crYOLO is a fast and accurate fully automated particle picker for cryo-EM. *Commun Biol* 2. DOI: [10.1038/s42003-019-0437-z](https://www.nature.com/articles/s42003-019-0437-z) diff --git a/wiki/Frequently-Asked-Questions.md b/wiki/Frequently-Asked-Questions.md deleted file mode 100644 index a940647..0000000 --- a/wiki/Frequently-Asked-Questions.md +++ /dev/null @@ -1,12 +0,0 @@ -# Frequently asked questions - -Here you can find frequently asked questions which appear from time to time when you process data with `TomoBEAR`: - -1. What can I do if `TomoBEAR` outputs `out of memory` error during execution of the module `DynamoTiltSeriesAlignment`? -* If you are running the module in parallel execution mode set the parameter `"execution_method"` in the `DynamoTiltSeriesAlignment` block to `"sequential"`. Then the execution will be slower because only one tilt stack will be processed at a time but the memory consumption will be less. - -2. What can I do if `TomoBEAR` outputs `out of memory` error during execution of the module `DynamoAlignmentProject`? -* This usually happens at low binning values (e.g., bin2 or bin1). You may put additional parameter `"dt_crop_in_memory": 0` to the corresponding `DynamoAlignmentProject` sections in order to prevent keeping the whole tomogram in the memory during processing. This will slow down processing because `Dynamo` will read tomogram volume several times but the memory consumption will be less. - -3. What can I do if `TomoBEAR` sorting files wrong for data collected using dose-symmetric scheme starting from 0-deg-tilt because my raw data files contain both `0.0` and `-0.0` tilts? For example, `TS_001_0.0_Sep07_19.38.12.tif` and `TS_002_-0.0_Sep07_19.52.42.tif`. -* This happens because physically zero angles are not ideally equal to zero, they are just very small (e.g., -0.0049795). To make sorting right in the described case you may add `"first_tilt_angle": 0` to `general` section of your input JSON file. diff --git a/wiki/Home.md b/wiki/Home.md index f6de14c..7b828e0 100644 --- a/wiki/Home.md +++ b/wiki/Home.md @@ -1,6 +1,6 @@ # Welcome to the TomoBEAR wiki! -![TomoBEAR Social Media Logo Image](/images/TomoBEAR_gitlogo.png) +![TomoBEAR Social Media Logo Image](https://github.com/KudryashevLab/TomoBEAR/blob/main/images/TomoBEAR_gitlogo.png) **B**asics for cryo-**E**lectron **tomo**graphy and **A**utomated **R**econstruction (**TomoBEAR**) is a configurable and customizable open-source MATLAB software package developed for automated large-scale parallelized cryo-electron tomography (cryo-ET) data processing. @@ -12,12 +12,11 @@ - [General description of the pipeline](#general-description) - [Installation and setup notes](https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup) -- [Frequently Asked Questions](https://github.com/KudryashevLab/TomoBEAR/wiki/Frequently-Asked-Questions) - [Tutorials on TomoBEAR setup and usage](https://github.com/KudryashevLab/TomoBEAR/wiki/Tutorials) - [Available modules description, their parameters and default values](https://github.com/KudryashevLab/TomoBEAR/wiki/Modules) - [Usage cases and tips](https://github.com/KudryashevLab/TomoBEAR/wiki/Usage) -- [List of external software to be cited](https://github.com/KudryashevLab/TomoBEAR/wiki/Additional-Software-Citation) - +- [Troubleshooting tips](https://github.com/KudryashevLab/TomoBEAR/wiki/Troubleshooting) +- [External software (installation, configuration, citation)](https://github.com/KudryashevLab/TomoBEAR/wiki/External-Software) ## Gerenal description @@ -33,41 +32,31 @@ Since number of **TomoBEAR** parameters is huge, to help users cope with that we
List of reasons to use TomoBEAR

-* Uses default presets which work on a variety of tested datasets -* Standalone and MATLAB versions are available -* Parallel execution is possible -* Computing resources are configurable -* Standardized folder structure +* Standalone and MATLAB versions +* Sequential and parallel execution with CPU- and GPU-enabled parallelization +* Standardized files and folders structure * Can deal with * misnumbered tilt images due to SerialEM crashes, based on timestamps * different naming conventions * EERs, MRCs and TIFs from K2, K3, Falcon4 * duplicated projections due to tracking issues (first, last, keep) * Restarting / resuming is possible (e.g. in case of errors, wrong configuration) - * Checkpoints are created after every processing step of a tilt series or tomogram -* Based on JSON configuration files which can be easily shared between others so that they can validate or improve your results -* Developed and tested on standard benchmarking datasets (such as EMPIAR-10064) to achieve same or better results as with manual processing -* You are able to look at the intermediates optimize parameters and rerun the steps to achieve optimal results -* You are never locked to the TomoBEAR processing pipeline and can easily breakout at various steps to other software tools you prefer -* Uses links where possible + * Checkpoints are created at the every processing step for the whole step and per tomogram +* Based on JSON configuration files (for sharing and reproducibility) +* Carefully designed preset of default parameter values +* Developed and tested on set of benchmarking datasets (EMPIAR-10064, EMPIAR-10452, EMPIAR-11543, EMPIAR-11306) +* Integration with IMOD and Dynamo projects * Clean up functionality to save storage * Tomograms to be processed can be limited to a subset * Uses Dynamo tilt-series alignment but injects the fiducial positions to IMOD for projection estimation -* Alignment parameters for tilt series can be (optionally) manually refined -* Routines for particle identification: template matching or geometry-assisted particle picking (with Dynamo) +* Routines for particles picking: template matching (modified Dynamo) or neural network based picking (crYOLO) * Improved Dynamo template matching functionality - * 10x - 14x speedup leveraging the GPU compared to 28 CPUs achieving 18x speedup -* Sub-stacking analysis (by SUSAN) is integrated (work in progress) -* TemplateMatchingPostprocessing - * Extraction of particles and conversion to dboxes to speed up file system access speed + * 10x speedup leveraging the GPU compared to 28 CPUs achieving 18x speedup + * Sub-stacks analysis (SUSAN) framework is integrated +* Particles extraction using Dynamo-based subtomogram cropping or SUSAN-based subtomogram reconstruction * DynamoAlignmentProject + * Generation of initial templates with with true structures and "noise traps" * Classification by multi-reference alignment (MRA) - * Generation of initial templates with with true structures and "noise traps" - * Extraction of particles with SUSAN and conversion to dboxes now possible - * Forcing of using SUSAN is also possible - * BinStacks module with appropriate binning levels should be run upfront - * Modification of particles box size is possible, as input can be used a factor, the absolute box size can be used as input -* Introduced presorting for single numbered tilt series data (as in Wenbo’s case of published Ryanodine receptor data) * Automated exclusion of bad tilts in reconstructions based on refined fiducial file from BatchRunTomo module

@@ -76,6 +65,6 @@ Since number of **TomoBEAR** parameters is huge, to help users cope with that we ## Pipeline structure In the following picture you can see a flow chart which visualizes pipeline steps which `TomoBEAR` can execute in an automated and parallel manner. -![Schematic Pipeline Image](/images/pipeline_light_mode.svg) +![Schematic Pipeline Image](https://github.com/KudryashevLab/TomoBEAR/blob/main/images/pipeline_light_mode.svg) Orange processing steps in the flow chart are mandatory and must be executed by TomoBEAR. Yellow boxes are optional and can be activated if desired. diff --git a/wiki/Installation-and-Setup.md b/wiki/Installation-and-Setup.md index 6049638..420b52b 100644 --- a/wiki/Installation-and-Setup.md +++ b/wiki/Installation-and-Setup.md @@ -3,54 +3,137 @@ > **Note** >
If you experience problems with TomoBEAR installation, do not hestitate to communicate with us by one of mentioned ways in [Feedback and Contribution](https://github.com/KudryashevLab/TomoBEAR#feedback-and-contribution) section. -# Prerequisites +## Contents -This software was developed and tested on machines with the following properties: +- [Prerequisites](#prerequisities) +- [Setup process overview](#setup-overview) +- [Video-tutorial link](#video-tutorial) +- [Step-by-step instructions](#detailed-instructions) -- Operating System (OS): CentOS 7 / Ubuntu 21.04 +## Prerequisites -Other Linux-based OSs should also be possible as long as MATLAB and all the other needed tools are runnable. +### Hardware -- Graphics Processing Unit (GPU): at least one GPU with a minimum of 8GB of Video Random Access Memory (VRAM) +1. **Graphics Processing Unit (GPU)**: at least one GPU with a minimum of 8GB of Video Random Access Memory (VRAM) It is better to have more GPUs with possibly greater amount of VRAM. With more GPUs you can process more data units in parallel. With more VRAM you can fit bigger tilt stacks or tomograms in memory especially in template matching. -- Random Access Memory (RAM): at least 16GB of RAM +2. **Random Access Memory (RAM)**: at least 16GB of RAM The more RAM you have the better. With more RAM you can run more parallel processes to process to process your data faster. This depends on the execution method of the modules. -- Hard Disk Drive (HDD) / Solid State Disk (SSD): depends on size of data +3. **Hard Disk Drive (HDD) / Solid State Disk (SSD)**: depends on size of the data being processed You need enough storage to store your data and all the intermediate data and results. Although it is possible to clean up the intermediate data during processing you will still need temporarily enough storage to store it until it can be deleted. The amount of needed storage is larger when you have more processes running in parallel. -# Setup +### Operating system -There are two ways to operate TomoBEAR. +The TomoBEAR was tested under the following Operating Systems (OS): **CentOS 7** / **Ubuntu 21.04**. -* The first way is to [[ use it directly from MATLAB | https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup#matlab ]] -* The second way is to [[ use a standalone executable which is available precompiled | https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup#standalone ]] +In general, any Linux-based OS's are suitable as long as MATLAB, CUDA and all the other needed for your processing external tools are installable and executable. -For both methods of operation, you will need to [[ get TomoBEAR source code | https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup#get-source-code-and-binary ]] and to get and install its dependencies. +### Middleware and software -You have to install essential TomoBEAR dependencies (CUDA, Dynamo, and IMOD) and may additionally install optional ones (MotionCor2, GCTF, etc.). The corresponding links you may find in the section [[ Additional software | https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup#additional-software ]]. +#### MATLAB + +The TomoBEAR was tested under the following MATLAB releases: **MATLAB R2021a**. + +There are two ways to operate TomoBEAR: + +* **(main) use it directly from MATLAB interactive session** - *currently it is better supported option* + +In this case you need to install the full version of MATLAB and obtain the required MATLAB license. Afterwards, the corresponding MATLAB executable should be added to your system's `PATH` variable. + +If you don't have administrator rights on your system, ask responsible person (laboratory technician or cluster/workstation administrator) to install it (recommended), otherwise you may try the next (in dev.) approach. + +* **(in dev.) use the standalone executable** provided along with some of the [source code releases](https://github.com/KudryashevLab/TomoBEAR/releases) + +If you would like to try TomoBEAR standalone version or don't have MATLAB license, you will need to use the **MATLAB Compiled Runtime (MRC)** libraries to be able to run compiled MATLAB binary. To configure it use the following instructions: + +1. In order to run TomoBEAR on the tested setup you need to get the newest available version of **MCR 2021a** [available here](https://www.mathworks.com/products/compiler/matlab-runtime.html). For example, you may get it directly to the selected folder using `wget` as the following: + + `wget https://ssd.mathworks.com/supportfiles/downloads/R2021a/Release/8/deployment_files/installer/complete/glnxa64/MATLAB_Runtime_R2021a_Update_8_glnxa64.zip` + +2. Further you need to give execution rights to the downloaded file using `chmod`. For that change to the folder where the file was downloaded to and execute the following command: + + `chmod u+x MATLAB_Runtime_R2021a_Update_8_glnxa64.zip` + +3. As the next step you need to extract the archive either with a command or through your file explorer, for example: + + `unzip MATLAB_Runtime_R2021a_Update_8_glnxa64.zip` + +4. Afterwards, you need change to the directory where the files were extracted, run the installation file and follow the installation wizard using GUI displayed on screen via + + `./install` + +5. Finally, when the installation is finished, remember to include the following paths of the installed MCR libraries to the environmental variable `LD_LIBRARY_PATH`: + + ```bash + /usr/local/MATLAB/MATLAB_Runtime/v910/runtime/glnxa64 + /usr/local/MATLAB/MATLAB_Runtime/v910/bin/glnxa64 + /usr/local/MATLAB/MATLAB_Runtime/v910/sys/os/glnxa64 + /usr/local/MATLAB/MATLAB_Runtime/v910/extern/bin/glnxa64 + ``` + +#### CUDA + +To benefit from GPU-enabled parallelization of some of the external software packages, the proper CUDA toolkits with the newest driver for your graphics card need to be installed. You need to get all the needed CUDA versions in your system installed to be able to run all the external tools which you need. + +> **Tipp** +>
Sometimes software which depends on the newer CUDA libraries can handle versions compiled with older CUDA libraries. Currently we recommend to install/use the **CUDA-11.5** as the most suitable version which complies with at least one of the available executables for each external tool using CUDA. + +To install CUDA you can use the package manager of your OS and install it manually or just use the module system of your facility/cluster/workstation if one is already employed. + +Alternatively, you may install CUDA manually. The easiest way for the manual installation is to add the repositories with CUDA to your specific OS package manager, otherwise may you download and run the corresponding to your OS executable available from the [NVIDIA homepage](https://developer.nvidia.com/cuda-toolkit-archive). + +The OS-specific tips: + +- **CentOS 7** + +The package manager is called `yum`. +A description of how to install a specific CUDA version for **CentOS 7** is [available here](https://linuxconfig.org/how-to-install-nvidia-cuda-toolkit-on-centos-7-linux). To get other CUDA versions you will need to replace the rpm `cuda-repo-rhel7-10.0.130-1.x86_64.rpm` with [other suitable rpm](https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/). + +- **Ubuntu 21.04 (or any other version)** + +To get the newest CUDA on an **Ubuntu** system the easiest way is to install it via graphical interface using *Software & Updates* (see tab *Additional Drivers*). Otherwise you may want to add CUDA repositories to the `apt` package manager. + +## Setup overview + +As it was mentioned above, there are two ways to operate TomoBEAR. + +* (main) use it directly from MATLAB interactive session - *currently it is better supported option* +* (in dev.) use the standalone executable provided along with some of the [source code releases](https://github.com/KudryashevLab/TomoBEAR/releases) + +For both methods of operation, to setup TomoBEAR you need to do the following: + + 0. Check prepresiquities. + 1. Get TomoBEAR source code (along with binary if needed). + 2. Get, install and configure all the mandatory and optional (if needed) dependencies. + 3. Configure and initialize TomoBEAR instance. + +Further on this page you may find the corresponding [video-tutorial](#video-tutorial) and [detailed step-wise instructions](#detailed-instructions). ## Video-tutorial We have prepared a small video-tutorial explaining the most basic `TomoBEAR` setup, which is needed for our [80S ribosome data processing tutorial](https://github.com/KudryashevLab/TomoBEAR/wiki/Tutorials): -* [Video-tutorial](https://drive.google.com/file/d/1Nhltdo7DbWdzPhAbnkzq7lCkDghxJrku/view?usp=share_link) explaining how to get the latest ```TomoBEAR``` version and configure ```TomoBEAR``` and its dependencies. +* [Video-tutorial](https://youtu.be/2uizkE616tE) explaining how to get the latest ```TomoBEAR``` version and configure ```TomoBEAR``` and its dependencies. -For more detailed or specific instructions please look at the further chapters provided on this page. +## Detailed instructions -## Get source code and binary +### Step 0. Check prerequisites -### Clone the latest version +Make sure that you comply with hardware and OS prerequisites, as well as that you have installed, accessible and executable middleware and software prerequisites before proceeding further. The corresponding instructions see in the [Prerequisites section](#prerequisites). + +### Step 1. Get source code and binary + +#### Clone the latest version To get the latest ```TomoBEAR``` version you can simply clone *main* branch by: ```bash git clone https://github.com/KudryashevLab/TomoBEAR.git ``` -### Clone specific version +#### Clone specific version To get a specific version of the ```TomoBEAR``` you need to create the following ```install_TomoBEAR.sh``` installation bash script: @@ -91,199 +174,75 @@ The list of available releases can be found on the [Releases page](https://githu > **Note** >
We are grateful to Wolfgang Lugmayr (CSSB, Hamburg) for the installation script idea. -## Initialize TomoBEAR - -### MATLAB - -If you want to run on a local machine then it is advised to run TomoBEAR from within MATLAB. This way you also don't need to download and install the MATLAB Compiled Runtime (MCR) if it is not already installed in your facility. - -It is adviced to use the following MATLAB release under which the software was tested: MATLAB R2021a. - -Further you need to install TomoBEAR's dependencies, at least all mandatory ones: CUDA, Dynamo and IMOD. For the corresponding links and some instructions on that refer to the section [Additional Software](https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup#additional-software). - -Afterwards you need to configure and initialize TomoBEAR with the corresponding paths to the dependencies. Instructions on that part you may find below in the subsection [Configure installed software](https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup#configure-installed-software) - -Finally, start MATLAB with the command - -* `./run_matlab.sh` - -To be able to run that MATLAB should be in your system's `PATH` variable. -The first time you start TomoBEAR after cloning it, initialization of necessary paths will be automatically performed, which could take a couple of minutes. But no worries, this is happening only once per the TomoBEAR clone you have fetched. - -### Standalone - -TomoBEAR can also be used as a standalone application. For that you will need to run the [installation script (see section above)](https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup#get-source-code-and-binary). -The available releases can be found on the [Releases page](https://github.com/KudryashevLab/TomoBEAR/releases). - -As well, you will need to install and setup the external software which is [mentioned below](https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup#additional-software). - -Additionally you will need the MCR (MATLAB Compiled Runtime) from [here](https://www.mathworks.com/products/compiler/matlab-runtime.html). There you need to get the newest **MCR 2021a** to be able to run TomoBEAR. When the download of the MCR is finished you will need to give it execution rights. -For that change to the folder where the file was downloaded to and execute the following command - -* `chmod u+x MATLAB_Runtime_R2021a_Update_4_glnxa64.zip` +### Step 2. Install and setup external software -Alternatively you can change to some folder and execute the following command before you execute the previous one +As TomoBEAR is wrapping standardized tools to fulfill some of the processing steps these need to be installed, executable and properly initialized in TomoBEAR. -* `wget https://ssd.mathworks.com/supportfiles/downloads/R2021a/Release/4/deployment_files/installer/complete/glnxa64/MATLAB_Runtime_R2021a_Update_4_glnxa64.zip` +You have to install essential TomoBEAR dependencies: **Dynamo** and **IMOD**. As well, you may additionally install optional external tools which you plan to use in TomoBEAR (**MotionCor2**, **GCTF**, etc.). -Afterwards you need to extract the archive either with a command or through your file explorer +Generally, you have two installation/configuration options: -* `unzip MATLAB_Runtime_R2021a_Update_4_glnxa64.zip` +1. Easier possibility for less experienced users: ask responsible for your cluster/workstation maintenance person to install the needed software and introduce that as modules. -Change to the directory where the files were extracted and run the installation with the following command and follow the wizard which is displayed on screen +2. Faster possibility for more experienced users: install the needed software on your own (use ```/home/username``` folder if you don't have root permissions) and configure it according to the corresponding instructions. -* `./install` +It is recommended to add the paths to the executables of the downloaded software in your `PATH` variable of your Linux system, except for **Dynamo** which must not be in the `PATH`. -When the installation is finished, remember to include to the enviromental variable `LD_LIBRARY_PATH` the following paths of the installed MCR libraries: - -```bash -/usr/local/MATLAB/MATLAB_Runtime/v910/runtime/glnxa64 -/usr/local/MATLAB/MATLAB_Runtime/v910/bin/glnxa64 -/usr/local/MATLAB/MATLAB_Runtime/v910/sys/os/glnxa64 -/usr/local/MATLAB/MATLAB_Runtime/v910/extern/bin/glnxa64 -``` - -Further you need to install TomoBEAR's dependencies, at least all mandatory ones: CUDA, Dynamo and IMOD. For the corresponding links and some instructions on that refer to the section [Additional Software](https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup#additional-software). - -Afterwards you need to configure and initialize TomoBEAR with the corresponding paths to the dependencies. Instructions on that part you may find below in the subsection [Configure installed software](https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup#configure-installed-software) - -Finally, initialize the current TomoBEAR standalone by running (from TomoBEAR code directory) - -* `./TomoBEAR init` +> **Warning** +>
+Since we have introduced a number of modifications to the original `Dynamo` code, which are distributed along with the `TomoBEAR`, you should not have original `Dynamo` codebase folder on your `PATH`, otherwise it will lead to the errors appearing during `Dynamo`-dependent modules execution. -which will perform the initialization of necessary paths automatically, which could take a couple of minutes. But this you should do only once per the TomoBEAR clone you have fetched. +The corresponding software versions, download links, installation notes and citation references you may find on the page [[ External software | https://github.com/KudryashevLab/TomoBEAR/wiki/External-Software ]]. -#### Configure installed software +### Step 3. Configure TomoBEAR defaults -Please, follow the setup instructions for all the software packages you downloaded. At the best, you will find the paths to the executables of the downloaded software in your `PATH` variable of your Linux system. +In this section you will configure TomoBEAR with the corresponding paths to the external tools available either as executables or via module system. -If this is not the case you need to adjust the paths to the executables in the +Further you need to adjust the paths to the executables in the `configurations/defaults.json` file. The keys where the values need to be adjusted can be found in the section `general` of the `defaults.json` file and are the following ones: * mandatory: - `"pipeline_location": ""` - location of the cloned TomoBEAR source code - `"dynamo_path": ""` - path to your Dynamo folder -* semi-mandatory (if you need motion/CTF-correction routines): +* semi-mandatory (motion/CTF-correction), when available as executables: - `"motion_correction_command": ""` - executable name of MotionCor2 or the full executable filepath - `"ctf_correction_command": ""` - executable name of Gctf/CTFFIND4 or the full executable filepath -* optional: +* optional, when available as executables: - `"aretomo_command": ""` - executable name of AreTomo or the full executable filepath - `"SUSAN_path": ""` - location of the cloned SUSAN source code - `"conda_path": ""` - location of the anaconda/miniconda +* optional, when available as modules: + - `"modules": ["IMOD_module", "Gctf_module", "MotionCor2_module", "CUDA_module_1", "CUDA_module_2"]` - insert the corresponding module names (will be loaded automatically by TomoBEAR). As for the other software packages you can add the required CUDA versions also to the field modules. -Additionaly, to be able to use IsoNet, you need fill in the following variables in the section `IsoNet`: -* `"isonet_env": ""` - name of the environment created and configured for IsoNet (following the [IsoNet setup instructions](https://github.com/IsoNet-cryoET/IsoNet) ) -* `"repository_path": ""` - location of the cloned IsoNet source code +Additionaly, to be able to use IsoNet/crYOLO, you need fill in the following variables in the section `IsoNet`/`crYOLO`: +* `"isonet_env": ""` or `"cryolo_env": ""` - name of the environment created and configured for IsoNet/crYOLO +* `"repository_path": ""` - location of the cloned IsoNet/crYOLO source code > **Warning** ->
Be sure to use a version that supports your current CUDA installation. - -> **Note** ->
Sometimes software which depends on the newer CUDA libraries can handle versions compiled with older CUDA libraries. +>
Be sure to use software versions that supports your current CUDA installation. However, sometimes software which depends on the newer CUDA libraries can handle versions compiled with older CUDA libraries. If you install IMOD the normal way then IMOD should be already in your `PATH` variable and therefore callable from everywhere. This is the only way of IMOD configuration supported by TomoBEAR. -If you use the Linux module system please insert the module names which need to be loaded to make all the necessary software available and working in the following variable in the general section - -* `"modules": ["IMOD_module", "Gctf_module", "MotionCor2_module", "CUDA_module_1", "CUDA_module_2"]` - -## Additional Software - -As TomoBEAR is wrapping standardized tools to fulfill some of the processing steps these need to be installed and executable. - -### Module System - -If you are working in a cryo electron microscopy facility and employ a cluster with a module system where all the needed software is already deployed as modules it is fairly easy to setup TomoBEAR. If not all the software packages are available as modules you have two possibilities. - -1. The first and probably the easiest possibility for inexperienced users is to ask the administrator or some responsible person for the module system to introduce the needed software as modules - -2. The second and probably faster possibility is to install the software on your own in your home folder if you don't have root permissions and put it your PATH variable or adjust the defaults.json so that the variables to tools contain the full path. - -If all the software is available as modules you need to head to the `defults.json` file and find the entry `"modules": []` just replace it with `"modules": ["IMOD_module", "Gctf_module", "MotionCor2_module", "CUDA_module_1", "CUDA_module_2"]`. Be aware that these module names are just placeholders for your real module names. you can find them out with the command `module available` or the shortcut `module avail`. - -As for the other software packages you can add the required CUDA versions also to the field modules. - -### Manual Installation - -The easiest way for the manual installation is to add the repositories with CUDA to your specific OS package manager. That is yum in CentOS and apt or apt-get in Ubuntu. The other way is a manual installation from the executables which are available from the [NVIDIA homepage](https://developer.nvidia.com/cuda-toolkit-archive). - -##### CUDA - -For all the additional software packages the proper CUDA toolkits with the newest driver for your graphics card need to be installed. - -To install CUDA you can use the package manager of your OS install it manually or just use the module system of your facility if you employ one. +### Step 4. Initialize TomoBEAR -##### CentOS 7 +#### MATLAB (interactive session) -A description of how to install a specific CUDA version for CentOS 7 is available [here](https://linuxconfig.org/how-to-install-nvidia-cuda-toolkit-on-centos-7-linux). Please follow the instructions there. To get other CUDA versions you will need to replace the rpm `cuda-repo-rhel7-10.0.130-1.x86_64.rpm` with a suitable rpm from [here](https://developer.download.nvidia.com/compute/cuda/repos/rhel7/x86_64/). +After you have installed the full licensed MATLAB version, go to the folder with cloned TomoBEAR source code and start MATLAB using the command -You need to repeat the steps multiple times until you have all the needed CUDA versions in ypur system installed to be able to run all the tools which are mentioned below. +`./run_matlab.sh` -##### Ubuntu 21.04 - -To get the newest CUDA on an Ubuntu system the easiest way is to install it via graphical interface using *Software & Updates* (see tab *Additional Drivers*). - -#### Dynamo - -For the non standalone version of TomoBEAR you need a Dynamo version with tilt stack alignment capabilities. The newest version can be [downloaded from here](https://wiki.dynamo.biozentrum.unibas.ch/w/index.php/Downloads). - -To minimize the dependencies on different CUDA versions it is advised to recompile the CUDA kernel for averaging with the newest CUDA version which is at best already available on your machine. If not please revise the chapter CUDA on this page. - -To recompile the kernel you just need to the location where dynamo was extracted and access the folder ```cuda``` inside. There you will find a file called ```makefile``` which you need to open and modify the second line containing the variable ```CUDA_ROOT=```. Please put in there the path to your most recent CUDA release available on the system. - -To recompile just execute the following two commands: - -* `make clean` -* `make all` - -#### MotionCor2 - -Head to the -[MotionCor2](https://docs.google.com/forms/d/e/1FAIpQLSfAQm5MA81qTx90W9JL6ClzSrM77tytsvyyHh1ZZWrFByhmfQ/viewform) -download page. There you need to register and download MotionCor2. A -MotionCor2 version greater than 1.4.0 is desired. - -- Alternative download [link](https://emcore.ucsf.edu/ucsf-software). - -#### AreTomo - -Head to the -[AreTomo](https://drive.google.com/drive/folders/1Z7pKVEdgMoNaUmd_cOFhlt-QCcfcwF3_) -download page. There you can find different AreTomo versions along with the documentation. - -#### Gctf - -You can download and try one of the following GCTF versions: -- [Gctf v1.06](https://www2.mrc-lmb.cam.ac.uk/download/gctf_v1-06-and-examples/) - the main version, tested to be working under TomoBEAR; -- [Gctf v1.18](https://www2.mrc-lmb.cam.ac.uk/download/special-version-for-phase-plate-gctf_v1-18/) - special version for phase plate data; -- [Gctf Gautomatch cu10.1](https://www2.mrc-lmb.cam.ac.uk/download/gctf_gautomatch_cu10-1-tar-gz/) - version for CUDA-10 (should work under TomoBEAR, but was not tested). - -#### CTFFIND4 - -Head to the [CTFFIND4](https://grigoriefflab.umassmed.edu/ctf_estimation_ctffind_ctftilt) or [cisTEM](https://cistem.org/) download page. There you can find CTFFIND4/cisTEM source code and/or binaries. - -> **Note** ->
In the following releases we are also planning to include CTFFIND4 updated version with tilted images support from the development version of the cisTEM package (https://cistem.org/development). - -#### IMOD - -Head to the -[IMOD](https://bio3d.colorado.edu/ftp/latestIMOD/RHEL7-64_CUDA8.0) -download page and get the IMOD version 4.10.42 or earlier. +> **Warning** +>
MATLAB should be in your system's `PATH` variable to be able to run the command above. -#### SUSAN +The first time you start TomoBEAR after cloning it, initialization of necessary paths will be automatically performed, which could take a couple of minutes. You need to initialize TomoBEAR this way only once per the TomoBEAR clone you have obtained. -To install and use SUSAN follow the instructions in the [SUSAN](https://github.com/rkms86/SUSAN) code repository. +#### Standalone (binary) -#### Anaconda +Initialize the current TomoBEAR standalone by running (from TomoBEAR code directory) -TomoBEAR can use various python based techniques to extend its functionality like using a neural net-based picker, denoising or missing wedge reconstruction algorithm. For that, it assumes you have Anaconda or Miniconda installed. For that either use your OS-included package manager or install it from the [Anaconda](https://www.anaconda.com/products/individual) web page. You can also take the miniconda installation to save on space and inodes. +`./TomoBEAR init` -#### IsoNet +which will perform the initialization of necessary paths automatically, which could take a couple of minutes. You need to initialize TomoBEAR this way only once per the TomoBEAR clone you have obtained. -IsoNet is a DL framework based on convolutional neural nets (CNNs) and the U-net architecture which can learn both to denoise and reconstruct missing wedge on cryo-elecrtron microscopy images. With TomoBEAR it is possible to perform those operations on tomograms using IsoNet. For that please clone the [IsoNet](https://github.com/IsoNet-cryoET/IsoNet) and follow the instructions on their page to setup Python environment. +## What is the next? -The original IsoNet version has limited missing wedge angular range of -60...+60, however there is an extended IsoNet version which allows for arbitrary parametrized missing wedge angular range on the ```mw_angle``` branch of the IsoNet original repository. TomoBEAR supports this version as well, so if you want to use it, you need to clone this branch by: -```bash -git clone --branch mw_angle https://github.com/IsoNet-cryoET/IsoNet.git IsoNet_mw -``` -where ```IsoNet_mw``` is the folder name where this IsoNet version will be cloned so that original IsoNet version is not overwritten if you already got one. +Upon completion of those setup steps you may proceed to [Tutorials page](https://github.com/KudryashevLab/TomoBEAR/wiki/Tutorials) to get idea of how to create input configuration files for your project and how to start TomoBEAR execution. diff --git a/wiki/Modules.md b/wiki/Modules.md index a9a4f34..c1cceea 100644 --- a/wiki/Modules.md +++ b/wiki/Modules.md @@ -4,12 +4,46 @@ and their parameters which can be setup in the JSON configuration file in their corresponding blocks. ## Contents +- [Modules dependency table](#modules-dependency-table) - [Global parameters configuration](#global-parameters-configuration) - [Pipeline behavior control modules](#pipeline-behavior-control-modules) - [CryoET data processing modules](#cryoet-data-processing-modules) -- [Template matching-associated modules](#template-matching-associated-modules) +- [Particles picking-associated modules](#particles-picking-associated-modules) - [Subtomogram Averaging modules](#subtomogram-averaging-modules) +## Modules dependency table + +Here we provide a table of modules dependencies on the external software. + +| Module \ Tool | IMOD | Dynamo | MotionCor2 | AreTomo | Gctf / CTFIND4 | IsoNet | crYOLO | SUSAN | Anaconda | +| :------------- | :--- | :---- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | +| MetaData | | | | | | | | | | +| SortFiles | | | | | | | | | | +| MotionCor2 | :white_check_mark: | | :white_check_mark: | | | | | | | +| GridEdgeEraser | | :white_check_mark: | | | | | | | | +| CreateStacks | :white_check_mark: | :white_check_mark: | | | | | | | +| DynamoTiltSeriesAlignment | | :white_check_mark: | | | | | | | | +| DynamoCleanStacks | :white_check_mark: | :white_check_mark: | | | | | | | +| AreTomo | :white_check_mark: | | | :white_check_mark: | | | | | | +| BatchRunTomo | :white_check_mark: | | | | | | | | | | +| GCTFCtfphaseflipCTFCorrection | :white_check_mark: | | | | :white_check_mark: | | | | | +| BinStacks | :white_check_mark: | | | | | | | | | +| Reconstruct | :white_check_mark: | | | | | | | | | +| IsoNet | (:white_check_mark:) | | | | | :white_check_mark: | | | :white_check_mark: | +| DynamoImportTomograms | | :white_check_mark: | | | | | | | | +| EMDTemplateGeneration | | :white_check_mark: | | | | | | | | +| TemplateGenerationFromFile | | :white_check_mark: | | | | | | | | +| DynamoTemplateMatching | | :white_check_mark: | | | | | | | | +| TemplateMatchingPostProcessing | | :white_check_mark: | | | | | | (:white_check_mark:) | | +| crYOLO | | (:white_check_mark:) | | | | | :white_check_mark: | | :white_check_mark: | +| GenerateParticles | | :white_check_mark: | | | | | | (:white_check_mark:) | | +| DynamoAlignmentProject | | :white_check_mark: | | | | | | (:white_check_mark:) | | + +Legend: +* :white_check_mark: - mandatory dependency +* (:white_check_mark:) - optional dependency +* empty square - not a dependency / tool is not used in the corresponding module + ## Global parameters configuration ### general @@ -512,7 +546,7 @@ This module performs grid edge identification and erases it for Au grids data. The CreateStacks module creates the stacks and normalizes them. There are two options for normalization. The default normalization scheme is to divide the projections by their frame count. TomoBEAR detects -automatically if you are using high-dose images and divides them by +automatically if you are using high-dose images from hybrid StA approach and divides them by their corresponding frame count in contrast to low-dose images where the frame-count is different. @@ -562,11 +596,8 @@ frame-count is different.
The DynamoTiltSeriesAlignment module is using the tilt stacks alignment -algorithm from dynamo which is the best available algorithm for -fiducial-based alignment. As default reasonable parameters for many cryo -ET projects are set. Some of them are dynamically derived. The option to -override non-dynamically derived parameters is still available and can -be done in the json configuration file. +algorithm from dynamo which is the state of the art available algorithm for fiducial-based alignment. As default, reasonable parameters for many cryo-ET projects are set. Some of them are dynamically derived. The option to +override non-dynamically derived parameters is available via the JSON configuration file. For troubleshooting and optimization of parameters it is possible to go to the processing folder and [use the Dynamo tools as in the Dynamo tutorial](https://wiki.dynamo.biozentrum.unibas.ch/w/index.php/Walkthrough_on_GUI_based_tilt_series_alignment). **Dependencies**
@@ -724,7 +755,7 @@ be done in the json configuration file. **Description**
The DynamoCleanStacks module can be run after the -DynamoTiltSeriesAlignment to automatically clean up the tilt stacks. For +DynamoTiltSeriesAlignment to automatically clean up the tilt stacks and to remove the projections where DynamoTiltSerieAlignment has not found gold beads. For that **DynamoCleanStacks** uses the output from dynamo tilt stacks alignment which states on which projections the fiducials could be fit. The others are then removed from the tilt stacks for further processing. @@ -760,7 +791,7 @@ The others are then removed from the tilt stacks for further processing. ### AreTomo **Description**
-This module performs AreTomo-based fiducial-free alignment. +The AreTomo module uses AreTomo to perform global or local fiducial-free alignment of the tilt stack. In order to optimize alignment parameters, please consult with the corresponding AreTomo documentation. **Dependencies**
@@ -772,24 +803,25 @@ This module performs AreTomo-based fiducial-free alignment.
```json - "AreTomo": { - "execution_method": "in_order", - "input_stack_binning": 1, - "reconstruction": false, - "weighted_back_projection": true, - "tilt_axis_refine_flag": 1, - "correct_tilt_axis_offset": 0, - "apply_given_tilt_axis_offset": false, - "tilt_axis_offset": 0, - "align_height_ratio": 0.75, - "apply_dose_weighting": false, - "sart": "20 5", - "roi": "0 0", - "roi_file": "", - "patch": "0 0", - "flip_volume": 1, - "flip_intensity": 0, - "citation": "" + "AreTomo": { + "execution_method": "in_order", + "input_stack_binning": 1, + "reconstruction": false, + "weighted_back_projection": true, + "tilt_axis_refine_flag": 1, + "correct_tilt_axis_offset": 0, + "apply_given_tilt_axis_offset": false, + "tilt_axis_offset": 0, + "align_height_ratio": 0.75, + "apply_dose_weighting": false, + "sart": "20 5", + "roi": "0 0", + "roi_file": "", + "patch": "0 0", + "flip_volume": 1, + "flip_intensity": 0, + "use_previous_alignment": false, + "citation": "" } ``` @@ -798,8 +830,7 @@ This module performs AreTomo-based fiducial-free alignment. **Description**
-The BatchRunTomo module is the most versatile one as it can be setup to -fulfill all the steps batchruntomo normally can do: +The BatchRunTomo is a versatile module performing IMOD operations. The detailed description can be found here: Tomography Guide for [IMOD Version 4.11](https://bio3d.colorado.edu/imod/doc/tomoguide.html). TomoBEAR can runs steps of batchruntomo: - 0: Setup - 1: Preprocessing @@ -984,9 +1015,7 @@ tomoBEAR and should not be replaced or changed by the user. **Description**
-The GCTFCtfphaseflipCTFCorrection module is detecting the CTF for the -tomograms which are reconstructed for template matching or particle -cropping. +The GCTFCtfphaseflipCTFCorrection module is detecting the defocus using Gctf or CTFIND4 on the tilt series which will be further CTF-corrected and used to reconstruct tomograms for template matching or particles generation. The results can be examined in the processing folders. As well, the module can be used for CTF-correction using ctfphaseflip from IMOD. **Dependencies**
@@ -1070,9 +1099,7 @@ cropping. ### BinStacks **Description**
-The BinStacks module is used for binning the stacks to be able to -reconstruct them with the Reconstruct module which should be used after -your stacks are binned. +The BinStacks module is used to bin tilt series stacks to be able to reconstruct them with the Reconstruct module. Stacks with selected binning levels will be produced. **Dependencies**
@@ -1100,6 +1127,7 @@ your stacks are binned. "BinStacks":{ "execution_method": "parallel", "use_ctf_corrected_aligned_stack": true, + "use_aligned_stack": false, "antialias_filter": 6, "defocus_tolerance": 20, "iWidth": 2, @@ -1118,11 +1146,7 @@ your stacks are binned. **Description**
-The Reconstruct module should be used after you binned the tilt stacks -with the BinStacks module or used aligned tilt stack binning option -greater than one. The module is set up by default to reconstruct binned -stacks. If you otherwise want to reconstruct unbinned stacks you need to -set up the Reconstruct module properly. +The Reconstruct module is used for tomogram reconstruction from aligned tilt stacks. The module is set up by default to reconstruct CTF-corrected binned stacks. If you otherwise want to reconstruct unbinned stacks or non-CTF-corrected stacks (for example, for further 3D CTF-deconvolution by IsoNet) you need to set up the Reconstruct module properly. **Dependencies**
@@ -1168,7 +1192,7 @@ set up the Reconstruct module properly. **Description**
-This module provides functionality of the IsoNet - DL framework for pre-processing, training and prediciton of restored information in the missing wedge region on the reconstructed cryo-elecrtron tomography volumes. +The IsoNet module provides functionality of the deep learning framework IsoNet used for reconstruction of the missing wedge (MW) region in tomograms. Besides, IsoNet provides an interface for denoising and 3D CTF-deconvolution. TomoBEAR interface includes such IsoNet routines as pre-processing (STAR file preparation, mask creation, CTF-deconvolution), training (refinement) and prediction. **Dependencies**
@@ -1181,7 +1205,7 @@ This module provides functionality of the IsoNet - DL framework for pre-processi Default parameters values (click to expand) ```json -"IsoNet": { + "IsoNet": { "execution_method": "once", "isonet_env": "", "repository_path": "", @@ -1263,16 +1287,16 @@ This module provides functionality of the IsoNet - DL framework for pre-processi
-## Template matching-associated modules +## Particles picking-associated modules ### DynamoImportTomograms **Description**
-The DynamoImportTomograms module generates a dynamo catalogue for you -and inputs the tomograms to that catalogue. After that you can call the -dynamo catalogue manager (dcm) to generate the models for the tomograms -or pick in them particles by hand. +The DynamoImportTomograms module generates a Dynamo Catalogue for user +and inputs the tomograms to that catalog. After that you can call the +Dynamo Catalogue Manager (dcm) to generate the models for the tomograms +or pick particles in them using the functionality of Dynamo Catalogue. **Dependencies**
@@ -1306,10 +1330,9 @@ or pick in them particles by hand. **Description**
The EMDTemplateGeneration module is used to automatically download a -EMDB template which is further down-scaled to match your desired template -matching binning. Besides that an automated routine to generate the mask -is also implemented. This module needs to be run before template -matching is executed. +EMDB template which is further down-scaled to match the requested template +matching binning level. Besides that an automated routine to generate the mask is also implemented. Either this module or the module TemplateGenerationFromFile needs to be run before template matching is executed by the DynamoTemplateMatching module. + **Dependencies**
@@ -1343,7 +1366,8 @@ matching is executed. **Description**
-The concept of this module is the same as in EMDTemplateMatching, with the only difference of taking the template from a user-defined path instead of fetching it directly from EMDB. +The TemplateGenerationFromFile module imports a map into the TomoBEAR workflow given by a path and scales it properly if the map header contains the correct pixel size; else the pixel size can be input as a parameter through the JSON-based configuration file. Either this module or the module EMDTemplateGeneration needs to be run before template matching is executed by the DynamoTemplateMatching module. + **Dependencies**
@@ -1377,9 +1401,8 @@ The concept of this module is the same as in EMDTemplateMatching, with the only **Description**
The DynamoTemplateMatching module re-implements the template -matching from Dynamo but on a GPU. Because of the GPU usage the whole -thing runs up to 15 times faster than the normal template matching -implementation of dynamo. +matching functionality from the Dynamo package, but distributes calculations on the GPU. The GPU-distributed version executes up to 12-15 times faster than the conventional CPU-distributed template matching +implementation available in Dynamo. In some multi-GPU systems the speed-up may be even non-linear relative to the number of GPUs used. **Dependencies**
@@ -1421,7 +1444,7 @@ implementation of dynamo. ### TemplateMatchingPostProcessing **Description**
-This module creates Dynamo-like table of particles based on the results of the template matching procedure from the module DynamoTemplateMatching. As well, using this module you can extract subtomograms of the identified particles. +The TemplateMatchingPostProcessing module takes the cross-correlation volumes generated by the DynamoTemplateMatching module and extracts the coordinates from the peaks found in them until a given threshold is reached. This threshold is set by default to a value of 2.5 standard deviations. As well this module could extract the corresponding subtomograms by cropping them from the reconstructed tomograms using Dynamo or reconstruct them directly from the aligned stacks using SUSAN. **Dependencies**
@@ -1474,12 +1497,80 @@ This module creates Dynamo-like table of particles based on the results of the t ``` -## Subtomogram Averaging modules +### crYOLO + +**Description** +
+The crYOLO module provides functionality of the deep learning framework crYOLO used for particle coordinate prediction. TomoBEAR interface includes such crYOLO routines as pre-processing (configuration file preparation, filtering), training and prediction. To optimize parameters of the training and prediction please use the corresponding crYOLO documentation (https://cryolo.readthedocs.io/en/stable/index.html). + +**Dependencies** +
+- Anaconda +- crYOLO (and its Python dependencies) +- CUDA + +
+
+ Default parameters values (click to expand) -### GenerateParticles.m +```json + "crYOLO": { + "execution_method": "once", + "cryolo_env": "", + "tomograms_to_use": [], + "steps_to_execute": {}, + "steps_to_execute_defaults": { + "config": { + "cryolo_command": "cryolo_gui.py config", + "target_boxsize": 220, + "config_json_filepath": "config_cryolo.json", + "filter": "LOWPASS", + "low_pass_cutoff": 0.1, + "janni_model_path": "", + "train_mode": false, + "train_tomograms_folder": "train_tomograms", + "train_tomograms_path": "", + "tomograms_binning": -1, + "train_annot_folder": "train_annot", + "input_size": -1 + }, + "train": { + "cryolo_command": "cryolo_train.py", + "config_json_filepath": "config_cryolo.json", + "num_cpu": -1, + "early": 15, + "warmup": 5 + }, + "predict": { + "cryolo_command": "cryolo_predict.py", + "config_json_filepath": "config_cryolo.json", + "trained_model_filepath": "cryolo_model.h5", + "threshold": 0.1, + "test_tomograms_folder": "test_tomograms", + "test_tomograms_path": "", + "tomograms_binning": -1, + "predict_annot_folder": "predict_annot", + "num_cpu": -1, + "tracing_search_range": -1, + "tracing_memory": -1, + "tracing_min_length": -1 + }, + "export_annotations": { + "raw_prtcl_coords_dir": "predict_annot/COORDS", + "per_table_particle_count": -1, + "total_particle_count": -1 + } + } + } +``` +
+ +### GenerateParticles **Description**
-This module uses Dynamo-like particles table to either extract Dynamo-like particles from the tomogram or to crop SUSAN-like substacks and make from them particles reconstructions. +The GenerateParticles module uses Dynamo-like particles tables to extract particles in one of the following ways: +* crop sub-tomograms from the reconstructed tomograms using Dynamo; +* reconstruct sub-tomograms from sub-stacks cropped directly from aligned tilt stacks using SUSAN. **Dependencies**
@@ -1509,12 +1600,13 @@ This module uses Dynamo-like particles table to either extract Dynamo-like parti ``` +## Subtomogram Averaging modules ### DynamoAlignmentProject **Description**
-This module is basically a wrapper for performing Dynamo alignment projects. +The DynamoAlignmentProject module can be set up to generate the two common Dynamo subtomogram averaging projects: multiple reference alignment (MRA) project and an independent half-set based refinement project. For classification we find it useful to have one or two classes with the correct reference that was used e.g. for template matching and some classes containing only noise (“noise traps”) which will attract suboptimal particles into classes for removal. **Dependencies**
diff --git a/wiki/Troubleshooting.md b/wiki/Troubleshooting.md new file mode 100644 index 0000000..0a01483 --- /dev/null +++ b/wiki/Troubleshooting.md @@ -0,0 +1,58 @@ +# Troubleshooting + +Here you can find troubleshooting tips and workarounds for `TomoBEAR`. + +### SUCESS and FAILURE control files + +After processing each module `TomoBEAR` puts processing control files `SUCCESS` or `FAILURE` depending on the outcome of the whole processing step. Additionally, for the modules which process tomograms independently of each other in parallel manner (like `CreateStacks`, `MotionCor2`, `BatchRunTomo`, etc.), inside each tomogram's folder there will be as well `SUCCESS` or `FAILURE` depending on the outcome of the particular tomogram processing. + +> **SUCCESS files contents** +>
+The file `SUCCESS` contains bit (0/1) vector of the number of tomograms length with 0s or 1s meaning failure or success of processing for the corresponding tomogram so that successful ones will be used on the further processing steps. This might be useful to know when you need workaround if something is not working. + +If you want to reprocess some particular step(s) or even particular tomogram(s) on certain steps, you need to delete the corresponding `SUCCESS`/`FAILURE` files, before restarting `TomoBEAR`. + +If for some reason you don't want to process further a certain tomogram, feel free to put in that tomogram's folder `FAILURE` file. + +Finally, if you put "tomogram_indices" in `"general"` section of the input JSON file and `TomoBEAR` still processing all of the tomograms for steps where they are processed independently, you might try to delete `SUCCESS` files for couple of previous step folders (but not in tomograms folders and not for steps which don't have tomogram folders, otherwise they would be reprocessed), so that `TomoBEAR` will just update its metadata properly. + +### Input Data Perception Issues + +- **In case your input data was collected using dose-symmetric scheme starting from 0-deg-tilt** + +If that is the case, the raw data files contain both `0.0` and `-0.0` tilts, for example, `TS_001_0.0_Sep07_19.38.12.tif` and `TS_002_-0.0_Sep07_19.52.42.tif`. This happens because physically zero angles are not ideally equal to zero, they are just very small (e.g., -0.0049795). To make sorting right in the described case you may add `"first_tilt_angle": 0` to `general` section of your input JSON file. + +- **In case your input data have more than one TS prefix** (separated by an underscore '_') + +We would recommend you to make symbolic links to the original raw data with a single TS prefix. +An example bash script ```softlink_files.sh``` to change prefix for all files of a single tilt serie in ```TIF``` data format is provided below: +```bash +prefix_to_cut=${1} +prefix_to_put=${2} +dir_data_raw=$(readlink -f ./${3}) +dir_data_links=$(readlink -f ./${4}) + +for file_tif_old in ${dir_data_raw}/${prefix_to_cut}*.tif ; do + file_tif_trunc=${file_tif_old#*${prefix_to_cut}} + file_mdoc_old=${file_tif_old}.mdoc + file_tif_new=${dir_data_links}/${prefix_to_put}$file_tif_trunc + file_mdoc_new=${file_tif_new}.mdoc + echo 'Linking ' $file_tif_old ' to ' $file_tif_new + ln -s $file_tif_old $file_tif_new + echo 'Linking ' $file_mdoc_old ' to ' $file_mdoc_new + ln -s $file_mdoc_old $file_mdoc_new +done +``` +You may use the script above ```softlink_files.sh``` to rename input data as softlinks to original files as the following: +```bash +softlink_files.sh PREFIX1_PREFIX2_PREFIX3_ PREFIXNEW_ original/dir/path renamed/dir/path +``` +where for each ```TIF``` file in the location ```original/dir/path``` a set of prefixes ```PREFIX1_PREFIX2_PREFIX3_``` will be substituted with a single prefix ```PREFIXNEW_``` and softlinks with the new filenames to the corresponding original files will be saved in the ```renamed/dir/path``` directory. + +### ERROR `out of memory` (`DynamoTiltSeriesAlignment`): + +If you are running the module in parallel execution mode set the parameter `"execution_method"` in the `DynamoTiltSeriesAlignment` block to `"sequential"`. Then the execution will be slower because only one tilt stack will be processed at a time but the memory consumption will be less. + +### ERROR `out of memory` (`DynamoAlignmentProject`): + +This usually happens at low binning values (e.g., bin2 or bin1). You may put additional parameter `"dt_crop_in_memory": 0` to the corresponding `DynamoAlignmentProject` sections in order to prevent keeping the whole tomogram in the memory during processing. This will slow down processing because `Dynamo` will read tomogram volume several times but the memory consumption will be less. diff --git a/wiki/Tutorials.md b/wiki/Tutorials.md index 7fc0345..60c1137 100644 --- a/wiki/Tutorials.md +++ b/wiki/Tutorials.md @@ -3,40 +3,40 @@ This page contains a range of tutorials explaining installation, setup, usage an ## Setup TomoBEAR First of all, you have to setup `TomoBEAR`: -* [Video-tutorial](https://drive.google.com/file/d/1Nhltdo7DbWdzPhAbnkzq7lCkDghxJrku/view?usp=share_link) explaining how to get the latest ```TomoBEAR``` version and configure ```TomoBEAR``` and its dependencies. +* [Video-tutorial](https://youtu.be/2uizkE616tE) explaining how to get the latest ```TomoBEAR``` version and configure ```TomoBEAR``` and its dependencies. -## Ribosome (EMPIAR-10064) +## 80S ribosome (EMPIAR-10064) -To provide an example of the `TomoBEAR` processing project we have chosen the well-known cryo-ET benchmarking dataset containing 80S ribosomes - EMPIAR-10064. +To provide an example of the `TomoBEAR` processing project we have chosen the well-known cryo-ET benchmarking data set containing 80S ribosomes - EMPIAR-10064. In this tutorial we will: * explain structure and important parameters of the input `JSON` file used as a configuration file in `TomoBEAR` projects; * explain output data structure of `TomoBEAR` projects; * explain pipeline execution and control by checkpoint files; -* guide you through the processing of the famous real-world dataset EMPIAR-10064 to achieve resolution of 11.25 Å with ~4k particles in nearly-automated parallelized manner. +* guide you through the processing of the famous real-world dataset EMPIAR-10064 to achieve resolution of 11.3 Å with 4003 particles in nearly-automated parallelized manner. -### Download tutorial data +#### Video-tutorials -You can download EMPIAR-10064 dataset here (`mixedCTEM` part): . After downloading the data extract it in a folder of your choice. +We have prepared a range of short (8-12 min) video-tutorials following the written version of the 80S ribosome tutorial provided below: -In our case we used just the `mixedCTEM` data and achieved 11.25 Å in resolution with ~4k particles which is similar to the resolution achieved by the original researchers. If you want you can additionally use the `CTEM` data to be able to pick even more particles. +* [Video-tutoral 1](https://youtu.be/N93tfAXp990): description of the project configuration file and the pipeline execution +* [Video-tutoral 2](https://youtu.be/qbkRtMJp0eI): additional configuration file parameters description, ```TomoBEAR```-```IMOD```-```TomoBEAR``` loop for checking tilt series alignment results and fiducials refinement (if needed); +* [Video-tutoral 3](https://youtu.be/BP2T_Y7BiDo): checking on further intermediate results (alignment, CTF-correction, reconstruction, template matching). -### Video-tutorials +### Step 1. Download tutorial data -We have prepared a range of short (8-12 min) video-tutorials following the written version of the 80S ribosome tutorial provided below: +You can download EMPIAR-10064 data set here (`mixedCTEM` part): . After downloading the data extract it in a folder of your choice. -* [Video-tutoral 1](https://drive.google.com/file/d/12h8Vs4eUeJDJAb5S6nF2BuOX7lfvmmYo/view?usp=share_link): description of the project configuration file and the pipeline execution -* [Video-tutoral 2](https://drive.google.com/file/d/16nNdu89XUZglO_PiRKmGkK85o_nWqOB8/view?usp=share_link): additional configuration file parameters description, ```TomoBEAR```-```IMOD```-```TomoBEAR``` loop for checking tilt series alignment results and fiducials refinement (if needed); -* [Video-tutoral 3](https://drive.google.com/file/d/1t3ol3KlHqBGtnZp1bnLHi_Anurd2VPcx/view?usp=share_link): checking on further intermediate results (alignment, CTF-correction, reconstruction, template matching). +In our case we used just the `mixedCTEM` data and achieved 11.3 Å in resolution with 4003 particles which is similar to the resolution achieved by the original researchers. If you want you can additionally use the `CTEM` data to be able to pick even more particles. -### Setup configuration file +### Step 2. Setup configuration file -In the `TomoBEAR` source code folder you will find a subfolder `configurations/` containing a file `ribosome_empiar_10064_dynamo.json`. This file describes the processing pipeline which should be setup by `TomoBEAR` to process used in this tutorial dataset. +In the `TomoBEAR` source code folder you will find a subfolder `configurations/` containing a file `ribosome_empiar_10064_dynamo.json`. This file describes the processing pipeline which should be setup by `TomoBEAR` to process used in this tutorial data set. The whole `JSON` file used for this tutorial you may also find at the end of this tutorial. The following paragraphs will explain the variables contained in the `JSON` file and the needed changes to be able to run `TomoBEAR` on your local machine. -First of all and most importantly you need to show `TomoBEAR` the path to the data and the processing folder. This must be done in the section `"general": {}` of the `JSON` file. +First of all and most importantly you need to show `TomoBEAR` the path to the data and the processing folder. This must be done in the section `"general": {}` of the `JSON` file, as shown below: ```json "general": { "project_name": "Ribosome", @@ -52,14 +52,14 @@ First of all and most importantly you need to show `TomoBEAR` the path to the da "binnings": [2, 4, 8], "reconstruction_thickness": 1400, "as_boxes": false - }, + } ``` -Since the pixel size and tilt angles are not provided in the header of MRC files of the used here dataset EMPIAR-10064, we have to add this information in the `"general": {}` section of the configuration file. +Since the pixel size and tilt angles are not provided in the header of MRC files of the used here data set EMPIAR-10064, we have to add this information in the `"general": {}` section of the configuration file. -### Execute until fiducials refinement +### Step 3. Execute until fiducials refinement -Everything else should be fine for now and the processing can be started. To run the `TomoBEAR` on the Ribosome data set you need to type in the following command in the command window of MATLAB +Everything else should be fine for now and the processing can be started. To run the `TomoBEAR` on the ribosome data set you need to type in the following command in the command window of MATLAB ```matlab runTomoBear("local", "/path/to/ribosome_empiar_10064_dynamo.json") @@ -91,9 +91,9 @@ When you followed all the steps thoroughly `TomoBEAR` should run up to the first ``` This can take a while, as the result of this segment `TomoBEAR` will create a folder structure with subfolders for the individual steps. You can monitor the progress of the execution in shell and by inspecting the contents of the folders. Upon success of an operation a file `SUCCESS` is written inside each folder. If you want to rerun a step you can terminate the process, change parameters, remove the `SUCCESS` file (or the entire subfolder) and restart the process. -Here the stacks have already been assembled, so neither `"Motioncorr2": {}`, not `"SortFiles": {}` modules were not needed. Here the key functionality is performed by `"DynamoTiltSeriesAlignment": {}` ([a recommended tutorial](https://wiki.dynamo.biozentrum.unibas.ch/w/index.php/Walkthrough_on_GUI_based_tilt_series_alignment)) after which the projections containing low number of tracked gold beads are excluded by `"DynamoCleanStacks": {}`. Finally, the output is converted into an IMOD project. The running time depends on your infrastructure and setup. +Here the stacks have already been assembled, so neither `"Motioncor2": {}`, nor `"SortFiles": {}` modules were not needed. Here the key functionality is performed by `"DynamoTiltSeriesAlignment": {}` ([a recommended tutorial](https://wiki.dynamo.biozentrum.unibas.ch/w/index.php/Walkthrough_on_GUI_based_tilt_series_alignment)) after which the projections containing low number of tracked gold beads are excluded by `"DynamoCleanStacks": {}`. Finally, the output is converted into an IMOD project. The running time depends on your infrastructure and setup. -### TomoBEAR-IMOD-TomoBEAR loop for fiducials refinement +### Step 4. TomoBEAR-IMOD-TomoBEAR loop for fiducials refinement After `TomoBEAR` stops you can inspect the fiducial model in the folder of `"BatchRunTomo": {}` which you can find in your processing folder. @@ -107,9 +107,13 @@ Now you can inspect the alignment of every tilt stack one after the other and ca etomo tomogram_xxx/*.edf ``` -When `etomo` starts just chose the `fine alignment` step which should be Lila if everything went fine for that tomogram and then click on `edit/view fiducial model` to start `3dmod` with the right options to be able to refine the gold beads. Before you start to refine just press the arrow up button in the top left corner of the window with the view port. To refine the gold beads click on `Go to next big residual` in the window with the stacked buttons from top to bottom and the view in the view port window should change immediately to the location of a gold bead with a big residual. Now see if you can center the marker better on the gold bead with the right mouse button. It is important that you don't put it on the peak of the red arrow but center it on the gold bead. When you are finished with this gold bead just press again on the `Go to next big residual` button. After you are finished with re-centering the marker on the gold beads you need to press the `Save and run tiltalign` button. +When `etomo` starts just chose the `fine alignment` step which should be magenta-colored if everything went fine for that tomogram and then click on `edit/view fiducial model` to start `3dmod` with the right options to be able to refine the gold beads. If you do not see the green circles - please go to the IMODimod command window, Edit -> Object l-> Type-> activate scattered and increase the size of the circle to e.g. 5. + +Before you start to refine just press the arrow up button in the top left corner of the window with the view port in order to zoom in. To refine the gold beads click on `Go to next big residual` in the window with the stacked buttons from top to bottom and the view in the view port window should change immediately to the location of a gold bead with a big residual. + +Now see if you can center the marker better on the gold bead with the right mouse button. It is important that you don't put it on the peak of the red arrow but center it on the gold bead. When you are finished with this gold bead just press again on the `Go to next big residual` button. After you are finished with re-centering the marker on the gold beads you need to press the `Save and run tiltalign` button. -### Execute up to tomogram reconstructions and template matching +### Step 5. Execute up to tomogram reconstructions and template matching After you finished the inspection of all the alignments you can start `TomoBEAR` again as previously and it will continue from where it stopped up to the next `StopPipeline` section. @@ -149,6 +153,8 @@ or if you are using a compiled version of `TomoBEAR` and have everything set up "flip_handedness": true }, "DynamoTemplateMatching": { + "sampling": 15, + "size_of_chunk": [463, 463, 175] }, "TemplateMatchingPostProcessing": { "cc_std": 2.5 @@ -157,9 +163,14 @@ or if you are using a compiled version of `TomoBEAR` and have everything set up This segment performs estimation of defocus, and hence, of the **Contrast Transfer Function** (**CTF**) using GCTF and subsequent CTF-correction using Ctfphaseflip from IMOD (`"GCTFCtfphaseflipCTFCorrection": {}`). You can inspect the quality of fitting by going into the folder `8_GCTFCtfphaseflipCTFCorrection_1` and typing `imod tomogram_xxx/slices/*.ctf` and making sure that the Thon rings match the estimation. If not - play with the parameters of the `GCTFCtfphaseflipCTFCorrection` module. -Then binned aligned CTF-corrected stacks are produced by `"BinStacks": {}` and tomographic reconstructions are generated for the binnings specified in the section `"general": {}`. In this example the particles are picked using template matching. First a template from EMDB is produced at a proper voxel size, then `"DynamoTemplateMatching": {}` creates **cross-correlation** (**CC**) volumes which can be inspected. Finally, highest cross-correlation peaks, over 2.5 standard deviations above the mean value in the cross-correlation volume are selected for extraction to 3D particle files, the initial coordinates are stored in the `particles_table` folder as a file in the dynamo table format. +Then binned aligned CTF-corrected stacks are produced by `"BinStacks": {}` and tomographic reconstructions are generated for the binnings specified in the section `"general": {}`. In this example the particles are picked using template matching. First a template from EMDB is produced at a proper voxel size, then `"DynamoTemplateMatching": {}` creates **cross-correlation** (**CC**) volumes which can be inspected. -### Execute subtomogram averaging (StA) +> **Note** +>
At a high binning level (e.g. 8 or 16) using the whole volume as a single chunk is more optimal than doing several chunks, so it is important to set the corresponding parameter to the size of the binned tomogram used for template matching. + +Finally, highest cross-correlation peaks, over 2.5 standard deviations above the mean value in the cross-correlation volume are selected for extraction to 3D particle files, the initial coordinates are stored in the `particles_table` folder as a file in the dynamo table format. + +### Step 6. Execute subtomogram averaging (StA) In the section below you will find **subtomogram classification projects** that should produce you a reasonable structure. They first use **multi-reference alignment projects** with a true class and so-called **noise trap classes** to first classify out false-positive particles produced by template matching, this happens at the binning which was used for template matching. In the end of the segment you should have a reasonable set of particles in the best class. @@ -238,7 +249,7 @@ Once you have selected the best class, insert corresponding class number in the ``` The section above is called **single reference project**, which will split the particles of the previously selected best class into two equally sized classes (called even/odd halves) with subsequent alignment of the particles in those halves to produce corresponding averages. This division will be needed further when unbinned data will be produced to be able to calculate the resolution of the resulting averaged map using **Fourier Shell Correlation** (**FSC**) curve. -After the first single reference project introduced above you will need to process tomograms by similar projects but at lower binnings in order to reduce the voxel size up to unbinned data to get the information corresponding to the highest possible resolution to be achieved using the current dataset. At this point automated workflow is finished as the user needs to play with the masks, particle sets, etc. +After the first single reference project introduced above you will need to process tomograms by similar projects but at lower binnings in order to reduce the voxel size up to unbinned data to get the information corresponding to the highest possible resolution to be achieved using the current data set. At this point automated workflow is finished as the user needs to play with the masks, particle sets, etc. You may want to try to use the following example of the end section of `JSON` file in order to have experience of processing tomograms at lower binnings to produce unbinned data to finally be able to calculate resolution of your ribosome electron-density map as a result of the first experience with `TomoBEAR`! @@ -276,7 +287,7 @@ You may want to try to use the following example of the end section of `JSON` fi If you get `out of memory` error while running some of `"DynamoAlignmentProject": {}` at lower binnings (especially the last one), you may put additional parameter `"dt_crop_in_memory": 0` to the corresponding `"DynamoAlignmentProject": {}` sections in order to prevent keeping the whole tomogram in memory for processing. For example, in this tutorial size of the one of unbinned tomograms is ~72Gb, while for binning 2 it is near 9Gb. -### Estimate resolution of the output final map +### Step 7. Estimate resolution of the output final map Finally, to estimate resolution of produced by `TomoBEAR` results, you need to use the following `Dynamo` command in `MATLAB`: ``` fsc = dfsc(path_to_half1, path_to_half2, 'apix', 2.62, 'mask', path_to_mask, 'show', 'on') @@ -288,7 +299,7 @@ After that you should get a similar FSC curve to the following one: ![Ribosome EMPIAR-10064 FSC](https://raw.githubusercontent.com/KudryashevLab/TomoBEAR/main/images/ribosome_empiar_10064_fsc.jpg) -where in Red we added a so-called "gold-standard" criterion of `FSC = 0.143` to estimate the final map resolution, which in our case for the final set of ~4k ribosome particles reached 11.25Å. +where in red we added a so-called "gold-standard" criterion of `FSC = 0.143` to estimate the final map resolution, which in our case for the final set of ~4k ribosome particles reached 11.3Å. ### Conclusion Here the Ribosome data set-based tutorial is finished. We thank you for trying out `TomoBEAR` and hope you have enjoyed it! @@ -348,6 +359,8 @@ Here the Ribosome data set-based tutorial is finished. We thank you for trying o "flip_handedness": true }, "DynamoTemplateMatching": { + "sampling": 15, + "size_of_chunk": [463, 463, 175] }, "TemplateMatchingPostProcessing": { "cc_std": 2.5 @@ -446,14 +459,131 @@ Here the Ribosome data set-based tutorial is finished. We thank you for trying o ``` +## In situ 80S ribosomes (EMPIAR-11306) + +As the second data set to showcase the capabilities of `TomoBEAR` we have chosen the [plasma-FIB-milled data set EMPIAR-11306](https://www.ebi.ac.uk/empiar/EMPIAR-11306/). + +This data set is fiducial-less and contains raw dose-fractionated movies in the Electron-Event Representation (EER) format. + +In our case we performed EER movies integration using MotionCor2 utilities, than we used IMOD-based patch-tracking in order to align this fiducial-less data set with the following CTF-estimation and correction and IMOD-based reconstruction. Finally, we used Dynamo-based template matching to pick particles. + +> **Note on StA part** +
Further StA processing happened outside of the TomoBEAR pipeline because of the tools used and a lot of manual intervention needed to accurately process this data set at the StA processing stage, for details see our preprint: +
Balyschew N, Yushkevich A, Mikirtumov V, Sanchez RM, Sprink T, Kudryashev M. Streamlined Structure Determination by Cryo-Electron Tomography and Subtomogram Averaging using TomoBEAR. **[Preprint]** 2023. bioRxiv doi: [10.1101/2023.01.10.523437](https://www.biorxiv.org/content/10.1101/2023.01.10.523437v1) + +Using the outlined approach we were able to achieve 6.2Å in resolution with ~18.3k final particles. The resolution from the [original publication](https://www.nature.com/articles/s41467-023-36372-9) by authors of this data set reached 4.9Å. +We suggest you to try to process this challenging data set, starting from the example configuration file given below. We find it as a useful exercise for both novice and advanced TomoBEAR users to improve the suggested workflow template to achieve better resolution. + +
+The full `JSON` configuration file to setup the processing pipeline and data in `TomoBEAR` (expand to see) + +```json +{ + "general": { + "project_name": "EMPIAR-11306", + "project_description": "Ribosome from FIB-milled data of EMPIAR-11306", + "data_path": "/path/to/raw/data/*.eer", + "processing_path": "/path/for/processing/", + "expected_symmetrie": "C1", + "aligned_stack_binning": 2, + "apix": 1.85, + "gpu": [x,x,x,x,x], + "binnings": [4, 8, 16], + "rotation_tilt_axis": 80, + "template_matching_binning": 8, + "as_boxes": false + }, + "MetaData": { + }, + "SortFiles": { + }, + "MotionCor2": { + "gain": "/path/to/reference/gain/20220406_175020_EER_GainReference.gain", + "eer_sampling": 2, + "eer_total_number_of_fractions": 343, + "eer_fraction_grouping": 34, + "eer_exposure_per_fraction": 0.5, + "ft_bin": 2 + }, + "CreateStacks": { + }, + "BatchRunTomo": { + "ending_step": 6, + "skip_steps": [5], + "directives": { + "comparam.xcorr.tiltxcorr.FilterRadius2": 0.07, + "comparam.xcorr.tiltxcorr.FilterSigma1": 0.00, + "comparam.xcorr.tiltxcorr.FilterSigma2": 0.02, + "comparam.xcorr.tiltxcorr.CumulativeCorrelation": 1, + "comparam.xcorr_pt.tiltxcorr.FilterRadius2": 0.07, + "comparam.xcorr_pt.tiltxcorr.FilterSigma1": 0.00, + "comparam.xcorr_pt.tiltxcorr.FilterSigma2": 0.02, + "comparam.xcorr_pt.tiltxcorr.IterateCorrelations": 1, + "comparam.xcorr_pt.tiltxcorr.SizeOfPatchesXandY": "500 500", + "comparam.xcorr_pt.tiltxcorr.OverlapOfPatchesXandY": "0.33 0.33", + "comparam.xcorr_pt.tiltxcorr.BordersInXandY": "102 102", + "runtime.Fiducials.any.trackingMethod": 1, + "runtime.AlignedStack.any.eraseGold": 0, + "comparam.align.tiltalign.RotOption": -1, + "comparam.align.tiltalign.MagOption": 0, + "comparam.align.tiltalign.TiltOption": 0, + "comparam.align.tiltalign.ProjectionStretch": 0 + } + }, + "BatchRunTomo": { + "starting_step": 8, + "ending_step": 8 + }, + "GCTFCtfphaseflipCTFCorrection": { + }, + "BatchRunTomo": { + "starting_step": 11, + "skip_steps": [12], + "ending_step": 13 + }, + "BinStacks": { + }, + "Reconstruct": { + "reconstruction_thickness": 3000, + "generate_exact_filtered_tomograms": true, + "exact_filter_size": 2500 + }, + "DynamoImportTomograms": { + }, + "TemplateGenerationFromFile": { + "template_path": "/path/to/template.mrc", + "mask_path": "/path/to/mask.mrc", + "template_apix": xx.x, + "use_smoothed_mask": false, + "use_bandpassed_template": false, + "use_ellipsoid": false + }, + "StopPipeline": { + }, + "DynamoTemplateMatching": { + "cone_range": 360, + "cone_sampling": 10, + "in_plane_range": 360, + "in_plane_sampling": 10, + "size_of_chunk": [512, 512, 375] + }, + "TemplateMatchingPostProcessing": { + "cc_std": 2.5, + "crop_particles": false + } +} +``` +
+ +In the `TomoBEAR` source code folder you will find a subfolder `configurations/` containing a file `pfib_ribosome_empiar_11306.json`. This file describes the processing pipeline which should be setup by `TomoBEAR` to process the suggested for this exercise data set. ## HIV1 (EMPIAR-10164) -As the second data set to showcase the capabilities of `TomoBEAR` we have chosen the HIV-1 data set with the number 10164 in the EMPIAR database. +As the third data set to showcase the capabilities of `TomoBEAR` we have chosen the HIV-1 data set [EMPIAR-10164](https://www.ebi.ac.uk/empiar/EMPIAR-10164/). -You can get the data set from [here](https://www.ebi.ac.uk/empiar/EMPIAR-10164/). In our case we use just the tomograms with the numbers 1, 3, 26, 28, 37 of the data and achieve 5.4Å in resolution with ~15.5k particles which is by now 1.5Å less than the resolution achieved by the original researchers. +In our case we use just the tomograms with the numbers 1, 3, 26, 28, 37 of the data and achieve 5.4Å in resolution with ~15.5k particles which is by now 1.5Å less than the resolution achieved by the original researchers. -After downloading the data extract it in a folder of your choice. One thing one should note about this data is that it is raw data. It is in the original form you acquire it from the microscope by SerialEM. +After downloading the data extract it in a folder of your choice. One thing one should note about these data is that it is raw data. It is in the original form you acquire it from the microscope by SerialEM. Following processing steps need to be applied to get tomograms @@ -461,3 +591,5 @@ Following processing steps need to be applied to get tomograms * the tilt stacks need to be assembled assembled * the tilt stacks need to be aligned * the tomograms need to be reconstructed + +In the `TomoBEAR` source code folder you will find a subfolder `configurations/` containing a file `hiv1_empiar_10164_dynamo.json`. This file describes the processing pipeline which should be setup by `TomoBEAR` to process the suggested for this exercise data set. diff --git a/wiki/Usage.md b/wiki/Usage.md index d2216e0..efab19a 100644 --- a/wiki/Usage.md +++ b/wiki/Usage.md @@ -1,8 +1,17 @@ # TomoBEAR Configuration -If you followed the [installation instructions](https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup) thoroughly then you should have a -working copy of the TomoBEAR. Now you need to generate a JSON file for your -project to start to process your acquired cryo-ET data. +If you followed the [installation instructions](https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup) thoroughly then you should have a working copy of the TomoBEAR. Now you need to generate a JSON file for your project to start to process your acquired cryo-ET data. + +## Contents + +- [Data entry](#data-entry) +- [Tilt stack alignment](#tilt-stack-alignment) +- [CTF estimation and correction](#ctf-estimation-and-correction) +- [Missing wedge reconstruction](#missing-wedge-reconstruction) +- [Particles picking](#particles-picking) +- [Particles generation](#particles-generation) +- [Live data processing](#live-data-processing) +- [Executing the workflow](#executing-the-workflow) ## Data entry @@ -29,38 +38,13 @@ Generally, TomoBEAR supports the ```TIF(F)```, ```MRC``` and ```EER``` input dat > (multi-shot data) >
In the case of multi-shot (e.g. [PACEtomo](https://github.com/eisfabian/PACEtomo)) input data it is important to make sure your data has described in the table above file format with leading keyword ```Square``` followed by the corresponding square number w/o any separators. -### Overcoming Input Data Perception Issues - The possible naming schemes for input data coming from SerialEM and defined above EER naming pattern should be all well-covered by our default regular expressions. Nevertheless, you may fine-tune your data perception by using the following parameters, if necessary: - ```_position``` where `````` could be ```prefix```, ```tomogram_number```, ```tilt_number```, ```angle```, ```date``` or ```time``` - this set of parameters directly controls recognition of the corresponding information by counting positions between underscores ```_``` in filenames and associating the extracted sub-strings with the corresponding keys; - ```_regex``` where `````` could be ```angle```, ```name```, ```number```, ```name_number``` or ```month_date``` - this set of parameters is more general and better tunable, it controls recognition of the corresponding information by regular expressions. -**In case your input data have more than one TS prefix**, separated by an underscore '_', and you experience issues with TomoBEAR filenames perception, we would recommend you to make symbolic links to the original raw data with a single TS prefix. -An example bash script ```softlink_files.sh``` to change prefix for all files of a single tilt serie in ```TIF``` data format is provided below: -```bash -prefix_to_cut=${1} -prefix_to_put=${2} -dir_data_raw=$(readlink -f ./${3}) -dir_data_links=$(readlink -f ./${4}) - -for file_tif_old in ${dir_data_raw}/${prefix_to_cut}*.tif ; do - file_tif_trunc=${file_tif_old#*${prefix_to_cut}} - file_mdoc_old=${file_tif_old}.mdoc - file_tif_new=${dir_data_links}/${prefix_to_put}$file_tif_trunc - file_mdoc_new=${file_tif_new}.mdoc - echo 'Linking ' $file_tif_old ' to ' $file_tif_new - ln -s $file_tif_old $file_tif_new - echo 'Linking ' $file_mdoc_old ' to ' $file_mdoc_new - ln -s $file_mdoc_old $file_mdoc_new -done -``` -You may use the script above ```softlink_files.sh``` to rename input data as softlinks to original files as the following: -```bash -softlink_files.sh PREFIX1_PREFIX2_PREFIX3_ PREFIXNEW_ original/dir/path renamed/dir/path -``` -where for each ```TIF``` file in the location ```original/dir/path``` a set of prefixes ```PREFIX1_PREFIX2_PREFIX3_``` will be substituted with a single prefix ```PREFIXNEW_``` and softlinks with the new filenames to the corresponding original files will be saved in the ```renamed/dir/path``` directory. +If you experience some problems with input data perception, take a look at the corresponding section on the [Troubleshooting page](https://github.com/KudryashevLab/TomoBEAR/wiki/Troubleshooting). -### EER input file format pre-processing +### Electron-Event Representation In order to be able to input and process ```EER``` files you have to setup MotionCor2 parameters of ```EER``` data integration into frame movies to be motion-corrected and integrated again to tilt images. Below you can find beginning of an example JSON configuration file for TomoBEAR: ```json @@ -99,189 +83,61 @@ where the parameters are the following: * ```eer_exposure_per_fraction```: total electron dose accumulated per frame (group of fractions). These parameters relate to the corresponding MotionCor2 parameters such as ```–EerSampling``` and three parameter values denoted in the file passed to ```–FmIntFile``` option, namely - total number of fractions (1st column), number of fractions in group (2nd column) and total accumulated dose per group (3rd column). -## JSON Configuration Templates - -### Raw Tomography Data with Fiducials +### Assembled tilt stacks -To process tomogrpahy data with fiducials the following template should -be used as configuration for the pipeline: -
- Example of full JSON file to process raw tomography data with fiducials (expand to see). +If you want to input already assembled tilt stacks with `TomoBEAR` you need to provide the tilt angles corresponding to the order of images in the input tilt stacks: ```json - { - "general": { - "project_name": "your project name", - "project_description": "your project name description", - "data_path": "/path/to/data/prefix*.mrc", - "processing_path": "/path/to/processing/folder", - "expected_symmetrie": "Cx", - "template_matching_binning": xx, - "gold_bead_size_in_nm": xx, - "reconstruction_thickness": xxxx, - "rotation_tilt_axis": xx, - "aligned_stack_binning": x, - "pre_aligned_stack_binning": x, - "binnings": [x, x, xx] - }, - "MetaData": { - }, - "SortFiles": { - }, - "MotionCor2": { - }, - "CreateStacks": { - }, - "DynamoTiltSeriesAlignment": { - }, - "DynamoCleanStacks": { - }, - "BatchRunTomo": { - "skip_steps": [4], - "ending_step": 6 - }, - "StopPipeline": { - }, - "BatchRunTomo": { - "starting_step": 8, - "ending_step": 8 - }, - "GCTFCtfphaseflipCTFCorrection": { - }, - "BatchRunTomo": { - "starting_step": 10, - "ending_step": 13 - }, - "BinStacks": { - }, - "Reconstruct": { - "reconstruct": "binned" - }, - "DynamoImportTomograms": { - }, - "EMDTemplateGeneration": { - "template_emd_number": "xxxx", - "template_bandpass_cut_off_resolution_in_angstrom": 20 - }, - "DynamoTemplateMatching": { - "size_of_chunk": [512, 720, 500], - "sampling": 15 - }, - "TemplateMatchingPostProcessing": { - "parallel_execution": false, - "cc_std": 2.5, - "crop_particles": true - } - } +{ + "general": { + "project_name": "your project name", + "project_description": "your project name description", + "data_path": "/path/to/data/prefix*.mrc", + "processing_path": "/path/to/processing/folder", + "expected_symmetrie": "Cx", + "tilt_angles": [-60.0, -58.0, -56.0, -54.0, -52.0, -50.0, -48.0, -46.0, -44.0, -42.0, -40.0, -38.0, -36.0, -34.0, -32.0, -30.0, -28.0, -26.0, -24.0, -22.0, -20.0, -18.0, -16.0, -14.0, -12.0, -10.0, -8.0, -6.0, -4.0, -2.0, 0.0, 2.0, 4.0, 6.0, 8.0, 10.0, 12.0, 14.0, 16.0, 18.0, 20.0, 0, 24.0, 26. , 28.0, 30.0, 32.0, 34.0, 36.0, 38.0, 40.0, 42.0, 44.0, 46.0, 48.0, 50.0, 52.0, 54.0, 56.0], + "apix": x.xx, + "reconstruction_thickness": xxxx, + "rotation_tilt_axis": xx, + "aligned_stack_binning": x, + "pre_aligned_stack_binning": x, + "binnings": [x, x, xx] + }, + "MetaData": { + }, + "CreateStacks": { + }, + ... +} ``` -
-
- -For some projects there are more sophisticted features available which can be switched on or fine tuned if needed. - -### Raw Tomography Data without Fiducials -To process tomogrpahy data without fiducials (for example, focused ion-beam milling data) user have the following options for fiducial-free alignment: -- BatchRunTomo-based (IMOD) patch tracking and alignment; -- AreTomo-based features tracking and alignment. +## Tilt stack alignment -
- BatchRunTomo-based (IMOD) template of JSON file to process raw tomogrpahy data without fiducials (expand to see). - -```json - { - "general": { - "project_name": "fibmil project", - "project_description": "fibmil project description", - "data_path": "/path/to/data/prefix*.tif", - "processing_path": "/path/to/processing/folder", - "gain": "/path/to/gain.mrc", - "dark": "/path/to/dark.mrc", - "expected_symmetrie": "Cx", - "template_matching_binning": xx, - "reconstruction_thickness": xxxx, - "rotation_tilt_axis": xx, - "aligned_stack_binning": x, - "pre_aligned_stack_binning": x, - "binnings": [x, x, xx], - "tilt_scheme": "bi_directional", - "tilt_angles": [-9, -6, -3, 0, 3, 6, 9, 12, 15, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48, 51, 54, 57, 60, -12, -15, -18, -21, -24, -27, -3033, -36, -39, -42, -45, -48, -51, -54, -57, -60] - }, - "MetaData": { - }, - "SortFiles": { - }, - "MotionCor2": { - }, - "CreateStacks": { - }, - "BatchRunTomo": { - "starting_step": 0, - "ending_step": 8, - "directives": { - "runtime.Fiducials.any.trackingMethod": 1, - "comparam.xcorr_pt.tiltxcorr.SizeOfPatchesXandY": "512,512", - "comparam.xcorr_pt.tiltxcorr.IterateCorrelations": 100, - "runtime.PatchTracking.any.adjustTiltAngles": 0, - "runtime.AlignedStack.any.eraseGold": 0, - "runtime.Positioning.any.hasGoldBeads": 0, - "comparam.xcorr_pt.tiltxcorr.FilterRadius2": 0.3, - "comparam.xcorr_pt.tiltxcorr.FilterSigma2": 0.4, - "comparam.xcorr_pt.tiltxcorr.OverlapOfPatchesXandY": "0.5,0.5" - } - }, - "GCTFCtfphaseflipCTFCorrection": { - }, - "BatchRunTomo": { - "starting_step": 10, - "ending_step": 13 - }, - "BinStacks": { - }, - "Reconstruct": { - "reconstruct": "binned" - }, - "StopPipeline": { - }, - "DynamoImportTomograms": { - }, - "EMDTemplateGeneration": { - "template_emd_number": "xxxx", - "template_bandpass_cut_off_resolution_in_angstrom": 20 - }, - "DynamoTemplateMatching": { - "sampling": 15 - }, - "TemplateMatchingPostProcessing": { - "cc_std": 2.5, - "crop_particles": true - } - } -``` -
-
+There is a couple of options available in TomoBEAR to perform tilt stacks alignment according to the fiducials presence in the sample: +* **fiducials-based alignment** - Dynamo/IMOD for fiducials model estimation, IMOD for alignment; +* **fiducials-free alignment** - IMOD for patches-based tracking and alignment or AreTomo for global or local ("smart grid") tracking and alignment. -For AreTomo usage case there is global alignment implemented in the pipeline as of ```TomoBEAR-v0.1.2```. In the future releases we are planning to enable as well local alignment procedure as well as AreTomo-based reconstructions. - -
- AreTomo-based template of JSON file to process raw tomogrpahy data without fiducials (expand to see). +### Fiducials-based alignment +To align tilt stacks containing fiducials the suggested `TomoBEAR` pipeline includes the following steps: +1. Coarsely pre-align tilt stacks using IMOD BatchRunTomo; +2. Estimate fiducials model using Dynamo Tilt Series Alignment; +3. Fine-align tilt stack by IMOD BatchRunTomo using obtained fiducials model from previous step. +To achieve that you need to use the configuration JSON file with the following structure: ```json - { +{ "general": { - "project_name": "fibmil project", - "project_description": "fibmil project description", + "project_name": "your project name", + "project_description": "your project name description", "data_path": "/path/to/data/prefix*.tif", "processing_path": "/path/to/processing/folder", - "gain": "/path/to/gain.mrc", - "dark": "/path/to/dark.mrc", "expected_symmetrie": "Cx", - "template_matching_binning": xx, - "reconstruction_thickness": xxxx, + "gold_bead_size_in_nm": xx, "rotation_tilt_axis": xx, + "pre_aligned_stack_binning": x, "aligned_stack_binning": x, "binnings": [x, x, xx], - "tilt_angles": [-9, -6, -3, 0, 3, 6, 9, 12, 15, 18, 21, 24, 27, 30, 33, 36, 39, 42, 45, 48, 51, 54, 57, 60, -12, -15, -18, -21, -24, -27, -3033, -36, -39, -42, -45, -48, -51, -54, -57, -60] + "apix": x.xx }, "MetaData": { }, @@ -291,526 +147,576 @@ For AreTomo usage case there is global alignment implemented in the pipeline as }, "CreateStacks": { }, - "AreTomo": { + "DynamoTiltSeriesAlignment": { }, - "StopPipeline": { + "DynamoCleanStacks": { }, - "GCTFCtfphaseflipCTFCorrection": { - "run_ctf_phase_flip": true + "BatchRunTomo": { + "skip_steps": [4], + "ending_step": 6 }, - "BinStacks": { + "StopPipeline": { }, - "Reconstruct": { - } - } -``` -
- -### Tilt Stacks with Fiducials - -If you want to process already assembled tilt stacks with `TomoBEAR` you -need to provide the tilt angles in the order in which they apear in the -tilt stacks. -
- Example of full JSON file to process assembled tilt stacks with fiducials (expand to see). - -```json - { - "general": { - "project_name": "Ribosome", - "project_description": "Ribosome Benchmark", - "data_path": "/sbdata/PTMP/nibalysc/ribosome/data/*.mrc", - "processing_path": "/sbdata/PTMP/nibalysc/ribosome", - "expected_symmetrie": "C1", - "tilt_scheme": "bi_directional", - "apix": 2.62, - "tilt_angles": [-60.0, -58.0, -56.0, -54.0, -52.0, -50.0, -48.0, -46.0, -44.0, -42.0, -40.0, -38.0, -36.0, -34.0, -32.0, -30.0, -28.0, -26.024.0, -22.0, -20.0, -18.0, -16.0, -14.0, -12.0, -10.0, -8.0, -6.0, -4.0, -2.0, 0.0, 2.0, 4.0, 6.0, 8.0, 10.0, 12.0, 14.0, 16.0, 18.0, 20.0, 0, 24.0, 26. , 28.0, 30.0, 32.0, 34.0, 36.0, 38.0, 40.0, 42.0, 44.0, 46.0, 48.0, 50.0, 52.0, 54.0, 56.0], - "gold_bead_size_in_nm": 9 - - }, - "MetaData": { - }, - "CreateStacks": { - "execution_method": "sequential" - }, - "DynamoTiltSeriesAlignment": { - "execution_method": "sequential" - }, - "DynamoCleanStacks": { - }, - "BatchRunTomo": { - "skip_steps": [4], - "ending_step": 6 - }, - "BatchRunTomo": { - "starting_step": 8, - "ending_step": 8 - }, - "GCTFCtfphaseflipCTFCorrection": { - }, - "BatchRunTomo": { - "starting_step": 10, - "ending_step": 13 - }, - "BinStacks": { - }, - "Reconstruct": { - }, - "DynamoImportTomograms": { - }, - "EMDTemplateGeneration": { - "template_emd_number": "3420" - }, - "DynamoTemplateMatching": { - }, - "TemplateMatchingPostProcessing": { - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_symmetrie": false, - "use_noise_classes": true - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_noise_classes": true, - "use_symmetrie": false, - "selected_classes": [1] - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_noise_classes": true, - "use_symmetrie": false, - "selected_classes": [1] - }, - "BinStacks": { - "use_ctf_corrected_aligned_stack": false, - "binnings": [2, 4, 8] - }, - "DynamoAlignmentProject": { - "classes": 1, - "iterations": 1, - "use_noise_classes": false, - "swap_particles": false, - "use_symmetrie": false, - "selected_classes": [1], - "as_boxes": true, - "use_SUSAN": true, - "binning": 8, - "threshold":0.75 - }, - "DynamoAlignmentProject": { - "classes": 1, - "iterations": 1, - "use_noise_classes": false, - "swap_particles": false, - "use_symmetrie": false, - "selected_classes": [1,2], - "as_boxes": true, - "use_SUSAN": true, - "binning": 4, - "threshold":0.75 - }, - "DynamoAlignmentProject": { - "classes": 1, - "iterations": 1, - "use_noise_classes": false, - "swap_particles": false, - "use_symmetrie": false, - "selected_classes": [1,2], - "as_boxes": true, - "use_SUSAN": true, - "binning": 2, - "threshold":0.75 - }, - "DynamoAlignmentProject": { - "classes": 1, - "iterations": 1, - "use_noise_classes": false, - "swap_particles": false, - "use_symmetrie": false, - "selected_classes": [1,2], - "as_boxes": true, - "use_SUSAN": true, - "binning": 1, - "threshold":0.75 - } - } + "BatchRunTomo": { + "starting_step": 8, + "ending_step": 8 + }, + ... +} ``` -
+If for some reason you want to use BatchRunTomo fiducials model estimation routines instead of Dynamo Tilt Series Alignment routines, feel free to remove from your configuration JSON file the following sections: `"DynamoTiltSeriesAlignment": {}` and `"DynamoCleanStacks": {}"`. -## Particles generation options +### Fiducials-free alignment -In TomoBEAR particles are automatically generated either after template matching on the `"TemplateMatchingPostProcessing"` step and during alignment/classification projects on the `"DynamoAlignmentProject"` steps. As well, if template matching cannot be used in a particular project, user may use ``"StopPipeline"`` module after ``"Reconstruction"`` step to pause processing, pick particles manually, put `Dynamo`-like table with picked particles coordinates and orientations in the `particles_table` folder in project's processing folder and resume processing. - -In `TomoBEAR` for automated particles generation you may choose one of the following tools to be used: -- `Dynamo` - in this case particles will be cropped from CTF-corrected tomograms; -- `SUSAN` - in this case particles will be reconstructed from individually CTF-corrected sub-stacks which were cropped from aligned non-CTF-corrected tilt stacks. - -By default `TomoBEAR` is using `Dynamo` for particles generation. In order to use `SUSAN`-base particles reconstruction add ``"use_SUSAN": true`` in particles-producing steps (`"DynamoAlignmentProject"` and/or `"TemplateMatchingPostProcessing"`) of your input JSON file. As well, for those steps you may change the following `SUSAN` parameters default values: - -```json - "ctf_correction_method": "defocus_file", - "susan_padding": 200, - "per_particle_ctf_correction": "phase_flip", - "padding_policy": "zero", - "normalization": "zm" -``` +To process tomogrpahy data without fiducials (for example, focused ion-beam milling data) user have the following options for fiducials-free alignment: -Please, remember that `SUSAN` needs non-CTF-corrected aligned stacks to be generated using `"BinStacks"` before the corresponding particles-generating modules will be used: -```json - "BinStacks": { - "binnings": [2, 4, 8], - "use_ctf_corrected_aligned_stack": false - } -``` +**1. BatchRunTomo-based (IMOD) patch tracking and alignment.**
- Example of full JSON file including SUSAN usage for particles generation for 80S ribosome dataset from tutorial (expand to see). + BatchRunTomo-based (IMOD) template of JSON file to process raw tomogrpahy data without fiducials (expand to see). ```json { "general": { - "project_name": "Ribosome", - "project_description": "Ribosome EMPIAR 10064", - "data_path": "/path/to/ribosome/data/*.mrc", + "project_name": "your project name", + "project_description": "your project name description", + "data_path": "/path/to/data/prefix*.tif", "processing_path": "/path/to/processing/folder", - "expected_symmetrie": "C1", - "apix": 2.62, - "tilt_angles": [-60.0, -58.0, -56.0, -54.0, -52.0, -50.0, -48.0, -46.0, -44.0, -42.0, -40.0, -38.0, -36.0, -34.0, -32.0, -30.0, -28.0, -26.0, -24.0, -22.0, -20.0, -18.0, -16.0, -14.0, -12.0, -10.0, -8.0, -6.0, -4.0, -2.0, 0.0, 2.0, 4.0, 6.0, 8.0, 10.0, 12.0, 14.0, 16.0, 18.0, 20.0, 22.0, 24.0, 26.0, 28.0, 30.0, 32.0, 34.0, 36.0, 38.0, 40.0, 42.0, 44.0, 46.0, 48.0, 50.0, 52.0, 54.0, 56.0, 58.0, 60.0], - "gold_bead_size_in_nm": 9, - "template_matching_binning": 8, - "reconstruction_thickness": 1400, - "rotation_tilt_axis": -5, - "as_boxes": false + "expected_symmetrie": "Cx", + "gold_bead_size_in_nm": xx, + "rotation_tilt_axis": xx, + "pre_aligned_stack_binning": x, + "aligned_stack_binning": x, + "binnings": [x, x, xx], + "apix": x.xx }, "MetaData": { }, - "CreateStacks": { - }, - "DynamoTiltSeriesAlignment": { - }, - "DynamoCleanStacks": { + "SortFiles": { }, - "BatchRunTomo": { - "skip_steps": [4], - "ending_step": 6 + "MotionCor2": { }, - "StopPipeline": { + "CreateStacks": { }, "BatchRunTomo": { - "starting_step": 8, - "ending_step": 8 - }, - "GCTFCtfphaseflipCTFCorrection": { - }, - "StopPipeline": { + "ending_step": 6, + "skip_steps": [5], + "directives": { + "comparam.xcorr_pt.tiltxcorr.SizeOfPatchesXandY": "500 500", + "comparam.xcorr_pt.tiltxcorr.OverlapOfPatchesXandY": "0.33 0.33", + "comparam.xcorr_pt.tiltxcorr.BordersInXandY": "102 102", + "runtime.Fiducials.any.trackingMethod": 1, + "runtime.AlignedStack.any.eraseGold": 0, + "comparam.align.tiltalign.RotOption": -1, + "comparam.align.tiltalign.MagOption": 0, + "comparam.align.tiltalign.TiltOption": 0, + "comparam.align.tiltalign.ProjectionStretch": 0 + } }, "BatchRunTomo": { - "starting_step": 10, - "ending_step": 13 + "starting_step": 8, + "ending_step": 8 }, - "BinStacks": { - "binnings": [8] +... +} +``` +except for base directives which you need to use in order to make patch-based alignment by BatchRunTomo we would advice to take a look at the following ones (for the usage please consult with the corresponding [IMOD documentation page](https://bio3d.colorado.edu/imod/doc/directives.html)): +```json +... + "directives": { + ... + "comparam.xcorr.tiltxcorr.FilterRadius2": 0.07, + "comparam.xcorr.tiltxcorr.FilterSigma1": 0.00, + "comparam.xcorr.tiltxcorr.FilterSigma2": 0.02, + "comparam.xcorr.tiltxcorr.CumulativeCorrelation": 1, + "comparam.xcorr_pt.tiltxcorr.FilterRadius2": 0.07, + "comparam.xcorr_pt.tiltxcorr.FilterSigma1": 0.00, + "comparam.xcorr_pt.tiltxcorr.FilterSigma2": 0.02, + "comparam.xcorr_pt.tiltxcorr.IterateCorrelations": 1, + ... + } +... +``` +
+
+ +**2. AreTomo-based features tracking and alignment.** + +For AreTomo usage case there is global and local alignment procedures implemented. In the future releases we are planning to enable as well AreTomo-based reconstructions. + +For AreTomo **global alignment** you need just to keep in mind the adjusted parameters values which you need to use for CTF-correction of the AreTomo globally aligned stack as in the provided example of configuration file below: +```json +{ + "general": { + "project_name": "your project name", + "project_description": "your project name description", + "data_path": "/path/to/data/prefix*.tif", + "processing_path": "/path/to/processing/folder", + "expected_symmetrie": "Cx", + "gold_bead_size_in_nm": xx, + "rotation_tilt_axis": xx, + "pre_aligned_stack_binning": x, + "aligned_stack_binning": x, + "binnings": [x, x, xx], + "apix": x.xx }, - "Reconstruct": { - "binnings": [8] + "MetaData": { }, - "DynamoImportTomograms": { + "SortFiles": { }, - "EMDTemplateGeneration": { - "template_emd_number": "3420", - "flip_handedness": true + "MotionCor2": { }, - "DynamoTemplateMatching": { + "CreateStacks": { }, - "BinStacks": { - "binnings": [2, 4, 8], - "use_ctf_corrected_aligned_stack": false - }, - "TemplateMatchingPostProcessing": { - "cc_std": 2.5, - "use_SUSAN": true, - "susan_padding": 40 - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_symmetrie": false, - "use_noise_classes": true - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_noise_classes": true, - "use_symmetrie": false, - "selected_classes": [1] - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_noise_classes": true, - "use_symmetrie": false, - "selected_classes": [1] - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_symmetrie": false, - "use_noise_classes": true, - "selected_classes": [1] - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_noise_classes": true, - "use_symmetrie": false, - "selected_classes": [1] - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_noise_classes": true, - "use_symmetrie": false, - "selected_classes": [1] - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 3, - "use_noise_classes": true, - "use_symmetrie": false, - "selected_classes": [1], - "box_size": 1.10, - "binning": 4, - "use_SUSAN": true, - "susan_padding": 40 + "AreTomo": { }, - "StopPipeline": { + "GCTFCtfphaseflipCTFCorrection": { + "run_ctf_phase_flip": true, + "use_aligned_stack": true }, - "DynamoAlignmentProject": { - "classes": 1, - "iterations": 1, - "use_noise_classes": false, - "swap_particles": false, - "use_symmetrie": false, - "selected_classes": [1], - "binning": 4, - "threshold":0.8 - }, - "DynamoAlignmentProject": { - "classes": 1, - "iterations": 1, - "use_noise_classes": false, - "swap_particles": false, - "use_symmetrie": false, - "selected_classes": [1,2], - "binning": 2, - "threshold":0.9, - "use_SUSAN": true, - "susan_padding": 40 - }, - "BinStacks":{ - "binnings": [1], - "use_ctf_corrected_aligned_stack": false - }, - "DynamoAlignmentProject": { - "classes": 1, - "iterations": 1, - "use_noise_classes": false, - "swap_particles": false, - "use_symmetrie": false, - "selected_classes": [1,2], - "binning": 1, - "threshold": 1, - "dt_crop_in_memory": 0, - "dynamo_allow_padding": 0, - "use_SUSAN": true, - "susan_padding": 40 - } + ... } ``` - -
- -As well, you may divide particles into so-called boxes (`Dynamo` dboxes-like batches) using ``"as_boxes": true`` (which is default) in ``"general"`` section of your JSON file: -- `Dynamo`: batch size is permanent and equal to 1000 prtcs/batch; -- `SUSAN`: batch size can be set by changing ``"susan_particles_batch"`` parameter default value (1000 prts/batch) in ``"general"`` section of an input JSON file. - -
- Example of full JSON file including SUSAN usage for particles generation with packaging particles in Dynamo-like boxes for 80S ribosome dataset from tutorial (expand to see). +For AreTomo **local alignment** you need to use the same way of CT-correction after using AreTomo section. Additionally, to speed up calculations and overcome artefacts we recommend to pre-bin data before AreTomo local alignment procedure. +Below you can find the example of the corresponding configuration file: ```json { "general": { - "project_name": "Ribosome", - "project_description": "Ribosome EMPIAR 10064", - "data_path": "/path/to/ribosome/data/*.mrc", + "project_name": "your project name", + "project_description": "your project name description", + "data_path": "/path/to/data/prefix*.tif", "processing_path": "/path/to/processing/folder", - "expected_symmetrie": "C1", - "apix": 2.62, - "tilt_angles": [-60.0, -58.0, -56.0, -54.0, -52.0, -50.0, -48.0, -46.0, -44.0, -42.0, -40.0, -38.0, -36.0, -34.0, -32.0, -30.0, -28.0, -26.0, -24.0, -22.0, -20.0, -18.0, -16.0, -14.0, -12.0, -10.0, -8.0, -6.0, -4.0, -2.0, 0.0, 2.0, 4.0, 6.0, 8.0, 10.0, 12.0, 14.0, 16.0, 18.0, 20.0, 22.0, 24.0, 26.0, 28.0, 30.0, 32.0, 34.0, 36.0, 38.0, 40.0, 42.0, 44.0, 46.0, 48.0, 50.0, 52.0, 54.0, 56.0, 58.0, 60.0], - "gold_bead_size_in_nm": 9, - "template_matching_binning": 8, - "reconstruction_thickness": 1400, - "rotation_tilt_axis": -5, - "as_boxes": true, - "susan_particle_batch": 2000 + "expected_symmetrie": "Cx", + "gold_bead_size_in_nm": xx, + "rotation_tilt_axis": xx, + "pre_aligned_stack_binning": x, + "aligned_stack_binning": x, + "binnings": [x, x, xx], + "apix": x.xx }, "MetaData": { }, - "CreateStacks": { - }, - "DynamoTiltSeriesAlignment": { + "SortFiles": { }, - "DynamoCleanStacks": { + "MotionCor2": { }, - "BatchRunTomo": { - "skip_steps": [4], - "ending_step": 6 + "CreateStacks": { }, - "StopPipeline": { + "BinStacks": { + "binnings": [2], + "use_ctf_corrected_aligned_stack": false, + "use_aligned_stack": false }, - "BatchRunTomo": { - "starting_step": 8, - "ending_step": 8 + "AreTomo": { + "input_stack_binning": 2, + "patch": "5 5" }, "GCTFCtfphaseflipCTFCorrection": { + "run_ctf_phase_flip": true, + "use_aligned_stack": true }, - "StopPipeline": { - }, + ... +} +``` + +## CTF estimation and correction + +For CTF-associated routines in TomoBEAR a number of popular in cryo-ET community options are available to correct CTF in tilt stacks and tomograms. + +### CTF routines for tilt stacks + +**CTF estimation** + +To estimate CTF in tilt stacks you need to use the module called `"GCTFCtfphaseflipCTFCorrection"`, which you can run with either `GCTF` or `CTFFIND4`. To choose the tool you want to use, setup the following parameters values: +* file: `defaults.json` > section: `"general"` > parameter: `"ctf_correction_command": ""` - provide the filename/path of the corresponding executable, for example: + * `"ctf_correction_command": "GCTF_v1.18_sm30-75_cu10.1"` + * `"ctf_correction_command": "ctffind"` +* file: `input.json` (your project configuration) > section: `"GCTFCtfphaseflipCTFCorrection"` > parameter: `"ctf_estimation_method": ""` - setup the corresponding method of CTF estimation: + * `"ctf_estimation_method": "gctf"` + * `"ctf_estimation_method": "ctffind"` + +You can inspect the quality of fitting by going into the folder `X_GCTFCtfphaseflipCTFCorrection_1` and typing `imod tomogram_xxx/slices/*.ctf` and making sure that the Thon rings match the estimation. If not - play with the parameters of the `GCTFCtfphaseflipCTFCorrection` module. + +**CTF correction** + +To correct CTF in tilt stacks in TomoBEAR we implemented Ctfphaseflip from IMOD which you may use: +* (e.g. before IMOD alignment) from `"BatchRunTomo"` by using step 11 of this module as the following + ```json "BatchRunTomo": { - "starting_step": 10, - "ending_step": 13 + "starting_step": 11, + "ending_step": 11 + } + ``` +* (e.g. before AreTomo alignment) from `"GCTFCtfphaseflipCTFCorrection"` by setting up `"run_ctf_phaseflip": true` + +### CTF routines for tomograms + +For CTF correction in tomograms in TomoBEAR you can use CTF-deconvolution procedure available as `IsoNet` module pre-processing functionality. For that you need first to get non-CTF-corrected aligned stacks and the corresponding non-CTF-corrected tomograms by the following sequence of configuration sections: +```json +... + "BinStacks": { + "binnings": [x, x, x], + "use_ctf_corrected_aligned_stack": false, + "use_aligned_stack": true }, + "Reconstruct": { + "binnings": [x, x, x], + "use_ctf_corrected_stack": false + }, +... +``` +and afterwards produce input STAR file and the target CTF-deconvolved tomograms by the following configuration section: +```json +... + "IsoNet": { + "isonet_env": "isonet-env", + "repository_path": "/path/to/cloned/IsoNet", + "steps_to_execute": { + "prepare_star": { + "tomograms_binning": x + }, + "deconv": { + "ncpu": x + } + } + }, +... +``` + +## Missing wedge reconstruction + +In order to fulfill interests of *in situ* cellular tomogaphy part of cryo-ET community, `TomoBEAR` interfaces `IsoNet` - DL framework capable of tomograms missing wedge (MW) reconstruction and denoising. `TomoBEAR` interface includes such `IsoNet` routines as preprocessing (STAR file preparation, mask creation, CTF-deconvolution), training (refinement) and prediction. + +### Pre-processing + +Since `IsoNet` requires quite good contrast to be able to restore missing wedge, it is recommended to use CTF-deconvolution as a preprocessing routine available in `IsoNet`. For that you need to get non-CTF-corrected aligned stacks and non-CTF-corrected tomograms before starting `IsoNet` by the following sequence of `TomoBEAR` modules: +```json +... "BinStacks": { - "binnings": [8] + "binnings": [x, x, x], + "use_ctf_corrected_aligned_stack": false, + "use_aligned_stack": true }, "Reconstruct": { - "binnings": [8] + "binnings": [x, x, x], + "use_ctf_corrected_stack": false }, - "DynamoImportTomograms": { +... +``` + +### Produce trained model and test it + +In order to be able to produce trained `IsoNet` model, you need to: +1. select 4-5 tomograms (the more different background is, the better training would be) +2. prepare STAR file listing the selected tomograms +3. (optional) deconvolve the selected tomograms +4. (optional) produce binary masks to train only on the selected regions of interest +5. extract 400-500 subtomograms (100-150 per tomogram) for training +6. start training procedure + +All of those steps could be covered just in one section of the corresponding `IsoNet` module as in the following example: +```json +... + "IsoNet": { + "isonet_env": "isonet-env", + "repository_path": "/path/to/cloned/IsoNet", + "tomograms_to_use": [x,xx,xx,xxx], + "steps_to_execute": { + "prepare_star": { + }, + "deconv": { + }, + "make_mask": { + }, + "extract": { + }, + "refine": { + "iterations": 30, + "noise_start_iter": [10,15,20,25], + "noise_level": [0.05,0.1,0.15,0.2] + } + } + }, +... +``` + +In case you do not want/need to produce binary masks for training, just skip the corresponding subsection `"make_mask"` in the `"steps_to_execute"` section. + +
+ In case you do not want/need to use IsoNet deconvolved tomograms (expand to see). + +Note that if for some reason you don't want to use `IsoNet` deconvolution procedure, you need to skip preprocessing procedure described in the previous subsection, skip `"deconv"` step in `"steps_to_execute"` section and set parameter `"use_ctf_corrected_tomograms": true` in the `"prepare_star"` subsection as shown in the example below: +```json +... + "IsoNet": { + "isonet_env": "isonet-env", + "repository_path": "/path/to/cloned/IsoNet", + "tomograms_to_use": [x,xx,xx,xxx], + "steps_to_execute": { + "prepare_star": { + "use_ctf_corrected_tomograms": true + }, + "make_mask": { + }, + "extract": { + }, + "refine": { + "iterations": 30, + "noise_start_iter": [10,15,20,25], + "noise_level": [0.05,0.1,0.15,0.2] + } + } + }, +... +``` +
+
+ +Once you produced trained `IsoNet` model, before starting prediction on all the tomograms you may want to test it first on the same tomograms which were used for training to asses training quiality and decide whether you need to repeat it with different parameters. In order to run MW prediction using trained model, you need the following module structure: +```json +... + "IsoNet": { + "isonet_env": "isonet-env", + "repository_path": "/path/to/cloned/IsoNet", + "tomograms_to_use": [x,xx,xx,xxx], + "steps_to_execute": { + "predict": { + "star_file": "../XX_IsoNet_1/tomograms.star", + "model": "../XX_IsoNet_1/results/model_iter30.h5" + } + } + }, +... +``` +where in `"tomograms_to_use": []` parameter you have to put exactly the same tomograms as those used for training, since you are going to use the already prepared STAR file from the training step. If you want to test it on other tomograms, use instructions given in the next section for the general prediction procedure. + +Other used parameters here are `"star_file": ""` and `"model": ""` where you have to put paths (relative to the current step folder or absolute) to the STAR file containing paths to deconvolved tomograms for prediction and to the trained `IsoNet` model. + +### Predict MW-free tomograms + +After coming up with suitable trained model, if you want to run prediction on all the data or the piece of data which differs from the one used for training, you need to prepare the corresponding STAR file and deconvolve it. You may achieve all of that by a the following configuration section: +```json +... + "IsoNet": { + "isonet_env": "isonet-env", + "repository_path": "/path/to/cloned/IsoNet", + "tomograms_to_use": [x,xx,xx,xxx], + "steps_to_execute": { + "prepare_star": { + }, + "deconv": { + }, + "predict": { + "model": "../XX_IsoNet_1/results/model_iter30.h5" + } + } }, +... +``` +In case you want to use all the tomograms for prediction, simply leave empty list in the corresponding field: `"tomograms_to_use": []`. + +## Particles picking + +TomoBEAR gives users a couple of particles picking options: +* template matching procedure by modified Dynamo routines (coordinates + orientations); +* neural network-based particles picking by crYOLO (coordinates only); + + +as well, not properly integrated in TomoBEAR yet, but as a workaround user may try +* manual particles picking (coordinates only). + +### Template matching + +In order to perform Dynamo-based template matching you need to +1. Prepare template + * by fetching a template directly from EMDB by ID, for example + ```json + ... "EMDTemplateGeneration": { - "template_emd_number": "3420", + "template_emd_number": "XXXX", "flip_handedness": true }, + ... + ``` + * by providing the path to the custom template file: + ```json + ... + "TemplateGenerationFromFile": { + "template_path": "/path/to/template.mrc", + "mask_path": "/path/to/mask.mrc", + "template_apix": xx.x + }, + ... + ``` +2. Run template matching search by cross-correlation (CC) + ```json + ... "DynamoTemplateMatching": { + "cone_range": 360, + "cone_sampling": 10, + "in_plane_range": 360, + "in_plane_sampling": 10, + "size_of_chunk": [xxx, xxx, xxx] + }, + ... + ``` + > **Note** + >
At a high binning level (e.g. 8 or 16) using the whole volume as a single chunk is more optimal than doing several chunks, so it is important to set the corresponding parameter to the size of the binned tomogram used for template matching. + +3. Post-process resulting CC map to get particles table (coordinates + orientations) + ```json + ... + "TemplateMatchingPostProcessing": { + "cc_std": 2.5 + }, + ... + ``` + +By default particles will be also cropped on-the-fly by Dynamo. If you want particles to be reconstructed by SUSAN instead of cropping them by Dynamo, you have to add parameter value `"use_SUSAN": true` to the `"TemplateMatchingPostProcessing"` section. However, if you do not want particles to be cropped at all, you have to add parameter value `"crop_particles": false` to the `"TemplateMatchingPostProcessing"` section. For more detailed instructions on particles generation procedure please refer to the **"Particles generation"** section below. + +### Neural network-based particles picking + +Another way to pick particles using `TomoBEAR` workflow is to use `crYOLO` - DL framework for particles coordinates prediction. `TomoBEAR` interface includes such `crYOLO` routines as preprocessing (config file preparation, filtering), training and prediction. + +#### Train data preparation + +For the train data preparation we recommend you to follow the original `crYOLO` tutorial on data preparation for [tomographic data crYOLO predictions](https://cryolo.readthedocs.io/en/stable/tutorials/tutorial_overview.html#tutorial-5-pick-particles-in-tomograms) (follow **Step 1. Data preparation**). + +#### Preprocessing and training + +After you created the needed training set of tomograms and corresponding annotations, you need to create the input configuration file. As well, it is highly recommended to filter tomograms in order to increase contrast and remove high-resolution noise, although it is an optional step. Both configuration and filtering are set up in one configuration subblock `"config"` along with the following training subblock `"train"` of the `"crYOLO"` configuration block of the input JSON configuration file, like in the example provided below: + +```json +... + "crYOLO": { + "tomograms_to_use": [x,xx,xxx], + "cryolo_env": "cryolo", + "steps_to_execute": { + "config": { + "target_boxsize": xx, + "filter": "LOWPASS", + "low_pass_cutoff": 0.3, + "train_mode": true, + "tomograms_binning": 8, + "train_annot_folder": "/path/to/dir/with/train/boxfiles" + }, + "train": { + "num_cpu": 10 + } + } }, - "BinStacks": { - "binnings": [2, 4, 8], - "use_ctf_corrected_aligned_stack": false - }, - "TemplateMatchingPostProcessing": { - "cc_std": 2.5, - "use_SUSAN": true, - "susan_padding": 40 - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_symmetrie": false, - "use_noise_classes": true - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_noise_classes": true, - "use_symmetrie": false, - "selected_classes": [1] - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_noise_classes": true, - "use_symmetrie": false, - "selected_classes": [1] - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_symmetrie": false, - "use_noise_classes": true, - "selected_classes": [1] - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_noise_classes": true, - "use_symmetrie": false, - "selected_classes": [1] - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 4, - "use_noise_classes": true, - "use_symmetrie": false, - "selected_classes": [1] - }, - "DynamoAlignmentProject": { - "iterations": 3, - "classes": 3, - "use_noise_classes": true, - "use_symmetrie": false, - "selected_classes": [1], - "box_size": 1.10, - "binning": 4, - "use_SUSAN": true, - "susan_padding": 40 +... +``` + +First of all, note that the `"config"` section of the example crYOLO step configuration section above contains `"train_mode": true` which indicates that configuration file is prepared for the train dataset (default is `false`). + +The `"target_boxsize"` parameter should be filled with the size of the box encapsulating target molecule in annotations prepared for training. + +The `"filter"` parameter controls whether the train tomograms will be filtered and the name of the filter to be used. Following the [original crYOLO convention and tutorial](https://cryolo.readthedocs.io/en/stable/tutorials/tutorial_overview.html#configuration), there are three types of values for this parameter: +* **"NONE"** - do not use any of the available filters (described below); +* **"LOWPASS"** - use low-pass filter (additional parameter `"low_pass_cutoff"` regulates low-passing threshold); +* **"JANNI"** - use JANNI denoising network: [pre-trained general model](https://sphire.mpg.de/wiki/doku.php?id=janni#janni_general_model) (additional parameter `"janni_model_path"` is for path to the pre-trained general model). + +In order to set the data to be used for training you may use the following parameters: +* `"train_annot_folder"` - path to the folder with annotation data for training (e.g. obtained via napari boxmanager plugin, as suggest by crYOLO authors) +* `"tomograms_binning"` - regulates binning of the tomograms to be used for training; + +By default, crYOLO module in TomoBEAR uses CTF-corrected tomograms of the binning level, corresponding to the `"tomograms_binning"` parameter value. However, it is better to increase/enhance contrast of the training dataset, because in that case prediction of target objects is more stable against false-positive picking of contamination, gold and empty membranes for noisy datasets. That is why usage of filters above is highly recommended. However, if you want to use tomograms enhanced/pre-processed by other method (for example, deconvolved or MW-restored by IsoNet), you may use parameter `"train_tomograms_path"` to set up path to the corresponding pre-processed tomograms. + +Section `"config"` is followed by section `"train"` to setup all the needed training parameters. + +The basic training setup is done and at this point `TomoBEAR` can be launched to train `crYOLO` model overnight. + +#### Predict particles positions + +After you got or trained by yourself `crYOLO` model, you may proceed to the prediction phase configuration. The example configuration file to obtain crYOLO particle coordinates predictions you may find below: +```json +... + "crYOLO": { + "tomograms_to_use": [x,xx,xxx], + "cryolo_env": "cryolo", + "steps_to_execute": { + "config": { + "target_boxsize": xx, + "filter": "LOWPASS", + "low_pass_cutoff": 0.3 + }, + "predict": { + "trained_model_filepath": "/path/to/trained/cryolo_model.h5", + "tomograms_binning": 8, + "num_cpu": 10 + } + } }, - "StopPipeline": { +... +``` +The structure is similar to the training stage: you need to prepare the configuration file by `"config"` and then use configuration subsection `"predict"`, setting up `"trained_model_path"` parameter value to obtain the particles positions as COORD or CBOX files. By default, as in the training section, the CTF-corrected tomograms on the corresponding binning level will be used as test tomograms for predictions. However, if you have need to provide alternative tomograms location (e.g. if you used this alternative location for training), you may set up additinoal parameter for the `"predict"` subsection called `"test_tomograms_path"`. + +Finally, to extract Dynamo-like particles table per each tomogram and for all tomograms together, you need to use the `"export_annotations"` subsection of the crYOLO module configuration as below: + +```json +... + "crYOLO": { + "cryolo_env": "cryolo", + "steps_to_execute": { + "export_annotations": { + "raw_prtcl_coords_dir": "/path/to/predict_annot/COORDS" + } + } }, - "DynamoAlignmentProject": { - "classes": 1, - "iterations": 1, - "use_noise_classes": false, - "swap_particles": false, - "use_symmetrie": false, - "selected_classes": [1], - "binning": 4, - "threshold":0.8 - }, - "DynamoAlignmentProject": { - "classes": 1, - "iterations": 1, - "use_noise_classes": false, - "swap_particles": false, - "use_symmetrie": false, - "selected_classes": [1,2], - "binning": 2, - "threshold":0.9, - "use_SUSAN": true, - "susan_padding": 40 - }, - "BinStacks":{ - "binnings": [1], - "use_ctf_corrected_aligned_stack": false - }, - "DynamoAlignmentProject": { - "classes": 1, - "iterations": 1, - "use_noise_classes": false, - "swap_particles": false, - "use_symmetrie": false, - "selected_classes": [1,2], - "binning": 1, - "threshold": 1, - "dt_crop_in_memory": 0, - "dynamo_allow_padding": 0, - "use_SUSAN": true, - "susan_padding": 40 - } -} +... ``` -
+ +### Manual particles picking + +In case template matching or NN-based solution cannot be used in a particular project, you may want to try manual picking procedure. + +Currently we do not directly integrate manual particles picking to TomoBEAR projects, but as a workaround user may use ``"StopPipeline"`` module after ``"Reconstruction"`` step to pause processing, pick particles manually, put or soft-link `Dynamo`-like table with picked particles coordinates and orientations in the `particles_table` folder in the project's processing folder and resume processing. + +## Particles generation + +In `TomoBEAR` particles could be generated by one of the following modules: +* directly using `"GenerateParticles"` +* automatically after template matching by the `"TemplateMatchingPostProcessing"` +* automatically between alignment/classification projects by the `"DynamoAlignmentProject"` + +In each case for automated particles generation you may choose one of the following tools to be used: +- `Dynamo` - in this case particles will be cropped from CTF-corrected tomograms; +- `SUSAN` - in this case particles will be reconstructed from individually CTF-corrected sub-stacks which were cropped from aligned non-CTF-corrected tilt stacks. + +By default `TomoBEAR` is using `Dynamo` for particles generation. In order to use `SUSAN`-base particles reconstruction add ``"use_SUSAN": true`` in particles-producing module of your choice (`"GenerateParticles"`, `"DynamoAlignmentProject"`, `"TemplateMatchingPostProcessing"`) in your input JSON file. As well, for those steps you may want to change the following `SUSAN` parameters default values: + +```json + "ctf_correction_method": "defocus_file", + "susan_padding": 200, + "per_particle_ctf_correction": "phase_flip", + "padding_policy": "zero", + "normalization": "zm" +``` + +Please, remember that `SUSAN` needs non-CTF-corrected aligned stacks to be generated using `"BinStacks"` before the corresponding particles-generating modules will be used: +```json +... + "BinStacks": { + "binnings": [2, 4, 8], + "use_ctf_corrected_aligned_stack": false, + "use_aligned_stack": true + }, +... +``` + +As well, you may divide particles into so-called boxes (`Dynamo` dboxes-like batches) using ``"as_boxes": true`` (which is default) in ``"general"`` section of your JSON file: +- `Dynamo`: batch size is permanent and equal to 1000 prtcs/batch; +- `SUSAN`: batch size can be set by changing ``"susan_particles_batch"`` parameter default value (1000 prts/batch) in ``"general"`` section of an input JSON file. ## Live data processing -**This is an experimental feature which is currently available only in development version under `develop_live` branch. Note that other `TomoBEAR` functionality in that brunch may differ from `main`.** + +**This is an experimental feature.** You may try out our new feature of the live data processing as it comes from the microscope! The main goal is to screen sample quality, e.g. identify presence of your molecular target in the sample. Main prerequisite for live data processing using `TomoBEAR` is an availability for `TomoBEAR` to access the data folder where collected files are appearing. -Currently, we have implemented "live" data processing for single-shot data collection mode. In this mode data is processed in "tilt serie by tilt serie" order. User needs to add two additional parameters to the input JSON file in the ``"general"`` section: +Currently, we have implemented "live" data processing for single-shot data collection mode. In this mode tilt series are processed as they are collected in the sequential order. User needs to add two additional parameters to the input JSON file in the ``"general"`` section: - `"minimum_files"` - number of files needed to consider tilt serie to be fully collected and subjected to processing (default: 15); - `"listening_time_threshold_in_minutes"` - threshold for a period of time passed from the latest arrived file of a tilt serie upon which that tilt serie would be considered as a fully collected and subjected to processing irregarding of the number of collected files (default: 15 [min]). @@ -869,13 +775,11 @@ In order to improve contrast in reconstructions we would recommend to enable `"g If you start collection from zero-tilt, to avoid problems of files perception and sorting caused by `-0.0`/`+0.0` appearing as the angle for untilted views (due to small initial tilting offset being present) instead of expected `0.0` it is also recommended to add `"first_tilt_angle": 0` to `general` section of your input JSON file. -# Executing the Workflow +## Executing the workflow -After you have generated an apropriate json file for your project you -may start the processing. There are several different execution strategies -which are described further in the following chapters. +After you have generated an appropriate JSON file for your project you may start the processing. There are several different execution strategies which are described further in the following chapters. -## Local Execution (Offline) +### Local Execution (Offline) To execute the workflow, you need to start MATLAB from the TomoBEAR cloned code folder using @@ -897,15 +801,16 @@ To run a workflow using a standalone TomoBEAR version, you need to go into the T This action assumes you have already configured everything according to the section [[ Standalone (Installation and Setup) | https://github.com/KudryashevLab/TomoBEAR/wiki/Installation-and-Setup#standalone ]]. -## Local Execution (Live) -**This is an experimental feature which is currently available only in development version under `develop_live` branch. Note that other `TomoBEAR` functionality in that brunch may differ from `main`.** +### Local Execution (Live) + +**This is an experimental feature!** To start `TomoBEAR` execution in live mode user need to type in the MATLAB command window: ```matlab runTomoBear("local_live", "/path/to/input.json", "/path/to/defaults.json") ``` -## SLURM Execution +### SLURM Execution For the execution of the workflow on a cluster you need to adjust the following keys in the general section: @@ -935,8 +840,8 @@ Or type the following command in the command window of MATLAB if you are using t ``` -## Cleanup -### Since 24-Nov-2022 +### Cleanup +#### Since 24-Nov-2022 To delete intermediate files in the `TomoBEAR` project folder you may use one of the following clean-up modes: - `cleanup` - in this case intermediate files will be deleted **only for those** processing steps where is used flag `"keep_intermediates": false`; @@ -953,7 +858,7 @@ If `"keep_intermediates": false` is used for some step, the corresponding files If user utilizes `cleanup_all` mode, deletion of all provided above files and folders may already save up to 30% of space initially occupied by the project folder. -#### Example 1 +##### Example 1 Let's consider you want to perform cleanup **only for** intermediate files associated with the step **MotionCor2** (i.e. delete files-folders listed in line 1 in the table above). Corresponding example of the JSON file for that cleanup setup is provided below: ```json @@ -1018,7 +923,7 @@ or in the shell using the following command: ./run_tomoBEAR.sh cleanup /path/to/project.json /path/to/defaults.json ``` -#### Example 2 +##### Example 2 Let's consider you want to perform cleanup **for all** intermediate files **except for** those which are associated with the step **GCTFCtfphaseflipCTFCorrection** (i.e. delete files/folders listed in lines 1-3 in the table above). Corresponding example of the JSON file for that cleanup setup is provided below: @@ -1084,7 +989,7 @@ or in the shell using the following command: ./run_tomoBEAR.sh cleanup_all /path/to/project.json /path/to/defaults.json ``` -### Before 24-Nov-2022 +#### Before 24-Nov-2022 If the later described **keep_intermediates** flag is set during the processing of a project to true you can cleanup the not needed