Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix endian bug and replace fileExtension with environmental variable omeconverter #565

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion formats/ome-converter-tool/.bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.3.2
current_version = 0.3.3-dev3
commit = True
tag = False
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\-(?P<release>[a-z]+)(?P<dev>\d+))?
Expand All @@ -24,6 +24,10 @@ replace = version = "{new_version}"

[bumpversion:file:VERSION]

[bumpversion:file:ict.yaml]

[bumpversion:file:omeconverter.cwl]

[bumpversion:file:README.md]

[bumpversion:file:src/polus/images/formats/ome_converter/__init__.py]
2 changes: 1 addition & 1 deletion formats/ome-converter-tool/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM polusai/bfio:2.3.3
FROM polusai/bfio:2.4.4

# environment variables defined in polusai/bfio
ENV EXEC_DIR="/opt/executables"
Expand Down
10 changes: 7 additions & 3 deletions formats/ome-converter-tool/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# OME Converter (v0.3.2)
# OME Converter (v0.3.3-dev3)

This WIPP plugin converts BioFormats supported data types to the
OME Zarr or OME TIF file format. This is not a complete implementation, rather it implements a file
Expand All @@ -23,12 +23,16 @@ contents of `plugin.json` into the pop-up window and submit.

## Options

This plugin takes 3 input arguments and 1 output argument:
This plugin takes 2 input arguments and 1 output argument:

| Name | Description | I/O | Type |
|------------------|--------------------------------------------------------------|--------|-------------|
| `--inpDir` | Input generic data collection to be processed by this plugin | Input | genericData |
| `--filePattern` | A filepattern, used to select data for conversion | Input | string |
| `--fileExtension`| A desired file format for conversion | Input | enum |
| `--outDir` | Output collection | Output | genericData |
| `--preview` | Generate a JSON file with outputs | Output | JSON |

## Docker Command

```bash
docker run -e POLUS_IMG_EXT=".ome.zarr" -v /Users/username/:/Users/username/ polusai/ome-converter-tool:0.3.3-dev3 --inpDir=/Users/path/to/Images/ --filePattern=".*.tif" --outDir=/Users/path/to/outputs
2 changes: 1 addition & 1 deletion formats/ome-converter-tool/VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.3.2
0.3.3-dev3
76 changes: 31 additions & 45 deletions formats/ome-converter-tool/ict.yaml
Original file line number Diff line number Diff line change
@@ -1,55 +1,41 @@
author:
- Nick Schaub
- Hamdah Shafqat
- Nick Schaub
- Hamdah Shafqat
contact: nick.schaub@nih.gov
container: polusai/ome-converter-tool:0.3.2-dev0
container: polusai/ome-converter-tool:0.3.3-dev3
description: Convert Bioformats supported format to OME Zarr or OME TIF
entrypoint: python3 -m polus.images.formats.ome_converter
inputs:
- description: Input generic data collection to be processed by this plugin
format:
- genericData
name: inpDir
required: true
type: path
- description: A filepattern, used to select data to be converted
format:
- string
name: filePattern
required: true
type: string
- description: Type of data conversion
format:
- enum
name: fileExtension
required: true
type: string
- description: Input generic data collection to be processed by this plugin
format:
- genericData
name: inpDir
required: true
type: path
- description: A filepattern, used to select data to be converted
format:
- string
name: filePattern
required: true
type: string
name: polusai/OMEConverter
outputs:
- description: Output collection
format:
- genericData
name: outDir
required: true
type: path
repository: https://github.com/PolusAI/polus-plugins
- description: Output collection
format:
- genericData
name: outDir
required: true
type: path
repository: https://github.com/PolusAI/image-tools
specVersion: 1.0.0
title: OME Converter
ui:
- description: Input generic data collection to be processed by this plugin
key: inputs.inpDir
title: Input generic collection
type: path
- description: A filepattern, used to select data for conversion
key: inputs.filePattern
title: Filepattern
type: text
- description: Type of data conversion
fields:
- .ome.tif
- .ome.zarr
- default
key: inputs.fileExtension
title: fileExtension
type: select
version: 0.3.2-dev0
- description: Input generic data collection to be processed by this plugin
key: inputs.inpDir
title: Input generic collection
type: path
- description: A filepattern, used to select data for conversion
key: inputs.filePattern
title: Filepattern
type: text
version: 0.3.3-dev3
6 changes: 1 addition & 5 deletions formats/ome-converter-tool/omeconverter.cwl
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
class: CommandLineTool
cwlVersion: v1.2
inputs:
fileExtension:
inputBinding:
prefix: --fileExtension
type: string
filePattern:
inputBinding:
prefix: --filePattern
Expand All @@ -24,7 +20,7 @@ outputs:
type: Directory
requirements:
DockerRequirement:
dockerPull: polusai/ome-converter-tool:0.3.2-dev0
dockerPull: polusai/ome-converter-tool:0.3.3-dev3
InitialWorkDirRequirement:
listing:
- entry: $(inputs.outDir)
Expand Down
26 changes: 3 additions & 23 deletions formats/ome-converter-tool/plugin.json
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
{
"name": "OME Converter",
"version": "0.3.2",
"version": "0.3.3-dev3",
"title": "OME Converter",
"description": "Convert Bioformats supported format to OME Zarr or OME TIF",
"author": "Nick Schaub (nick.schaub@nih.gov), Hamdah Shafqat Abbasi (hamdahshafqat.abbasi@nih.gov)",
"institution": "National Center for Advancing Translational Sciences, National Institutes of Health",
"repository": "https://github.com/PolusAI/polus-plugins",
"repository": "https://github.com/PolusAI/image-tools",
"website": "https://ncats.nih.gov/preclinical/core/informatics",
"citation": "",
"containerId": "polusai/ome-converter-tool:0.3.2",
"containerId": "polusai/ome-converter-tool:0.3.3-dev3",
"baseCommand": [
"python3",
"-m",
Expand All @@ -26,20 +26,6 @@
"type": "string",
"description": "A filepattern, used to select data to be converted",
"required": true
},
{
"name": "fileExtension",
"type": "enum",
"description": "Type of data conversion",
"default": "default",
"options": {
"values": [
".ome.tif",
".ome.zarr",
"default"
]
},
"required": true
}
],
"outputs": [
Expand All @@ -59,12 +45,6 @@
"key": "inputs.filePattern",
"title": "Filepattern",
"description": "A filepattern, used to select data for conversion"
},
{
"key": "inputs.fileExtension",
"title": "fileExtension",
"description": "Type of data conversion",
"default": ".ome.tif"
}
]
}
4 changes: 2 additions & 2 deletions formats/ome-converter-tool/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "polus-images-formats-ome-converter"
version = "0.3.2"
version = "0.3.3-dev3"
description = "Convert BioFormats datatypes to ome.tif or ome.zarr file format"
authors = [
"Nick Schaub <nick.schaub@nih.gov>",
Expand All @@ -12,7 +12,7 @@ packages = [{include = "polus", from = "src"}]

[tool.poetry.dependencies]
python = ">=3.9,<3.12"
bfio = {version = "^2.3.3", extras = ["all"]}
bfio = {version = "^2.4.4", extras = ["all"]}
filepattern = "^2.0.4"
typer = "^0.7.0"
tqdm = "^4.64.1"
Expand Down
7 changes: 3 additions & 4 deletions formats/ome-converter-tool/run-plugin.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,11 @@ fileExtension=".ome.zarr"
outDir=/data/output

# Show the help options
docker run polusai/ome-converter-plugin:${version}
# docker run polusai/ome-converter-plugin:${version}

# Run the plugin
docker run --mount type=bind,source=${datapath},target=/data/ \
polusai/ome-converter-plugin:${version} \
docker run -e POLUS_IMG_EXT=${fileExtension} --mount type=bind,source=${datapath},target=/data/ \
polusai/ome-converter-tool:${version} \
--inpDir ${inpDir} \
--filePattern ${filePattern} \
--fileExtension ${fileExtension} \
--outDir ${outDir}
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""Ome Converter."""

__version__ = "0.3.2"
__version__ = "0.3.3-dev3"

from .image_converter import batch_convert
from .image_converter import convert_image
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
import preadator
import typer
from polus.images.formats.ome_converter.image_converter import NUM_THREADS
from polus.images.formats.ome_converter.image_converter import Extension
from polus.images.formats.ome_converter.image_converter import convert_image
from tqdm import tqdm

Expand All @@ -24,6 +23,7 @@
)
logger = logging.getLogger("polus.images.formats.ome_converter")
logger.setLevel(os.environ.get("POLUS_LOG", logging.INFO))
POLUS_IMG_EXT = os.environ.get("POLUS_IMG_EXT", ".ome.tif")


@app.command()
Expand All @@ -43,11 +43,6 @@ def main(
"--filePattern",
help="A filepattern defining the images to be converted",
),
file_extension: Extension = typer.Option(
Extension,
"--fileExtension",
help="Type of data conversion",
),
out_dir: pathlib.Path = typer.Option(
...,
"--outDir",
Expand All @@ -68,7 +63,6 @@ def main(
logger.info(f"inpDir = {inp_dir}")
logger.info(f"outDir = {out_dir}")
logger.info(f"filePattern = {pattern}")
logger.info(f"fileExtension = {file_extension}")

fps = fp.FilePattern(inp_dir, pattern)

Expand All @@ -79,7 +73,7 @@ def main(
"outDir": [],
}
for file in fps():
out_name = str(file[1][0].name.split(".")[0]) + file_extension
out_name = str(file[1][0].name.split(".")[0]) + POLUS_IMG_EXT
out_json["outDir"].append(out_name)
json.dump(out_json, jfile, indent=2)
return
Expand All @@ -93,14 +87,14 @@ def main(
for files in fps():
file = files[1][0]
threads.append(
executor.submit_process(convert_image, file, file_extension, out_dir),
executor.submit_process(convert_image, file, POLUS_IMG_EXT, out_dir),
)

for f in tqdm(
as_completed(threads),
total=len(threads),
mininterval=5,
desc=f"converting images to {file_extension}",
desc=f"converting images to {POLUS_IMG_EXT}",
initial=0,
unit_scale=True,
colour="cyan",
Expand Down
Loading
Loading