Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardized names of all updated tools. #518

Merged
merged 8 commits into from
Feb 16, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 0 additions & 1 deletion clustering/k-means-clustering-plugin/VERSION

This file was deleted.

4 changes: 0 additions & 4 deletions clustering/k-means-clustering-plugin/build-docker.sh

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.3.4-dev2
current_version = 0.3.5-dev0
commit = True
tag = False
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\-(?P<release>[a-z]+)(?P<dev>\d+))?
Expand Down Expand Up @@ -28,4 +28,4 @@ replace = version = "{new_version}"

[bumpversion:file:CHANGELOG.md]

[bumpversion:file:src/polus/plugins/clustering/k_means/__init__.py]
[bumpversion:file:src/polus/images/clustering/k_means/__init__.py]
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# K-Means Clustering(0.3.4-dev2)
# K-Means Clustering(0.3.5-dev0)

1. This plugin is updated only to the new plugin standards
2. Before plugin support only `.csv` as an input files supported `.csv` and `.feather` file formats. Now this plugin support other vaex supported file formats both as inputs and outputs.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@ COPY src ${EXEC_DIR}/src

RUN pip3 install ${EXEC_DIR} --no-cache-dir

ENTRYPOINT ["python3", "-m", "polus.plugins.clustering.k_means"]
ENTRYPOINT ["python3", "-m", "polus.images.clustering.k_means"]
CMD ["--help"]
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# K-Means Clustering(v0.3.4-dev2)
# K-Means Clustering(v0.3.5-dev0)

The K-Means Clustering plugin clusters the data using Scikit-learn K-Means clustering algorithm and outputs csv file. Each instance(row) in the input csv file is assigned to one of the clusters. The output csv file contains the column 'Cluster' that shows which cluster the instance belongs to.

Expand Down
1 change: 1 addition & 0 deletions clustering/k-means-clustering-tool/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0.3.5-dev0
4 changes: 4 additions & 0 deletions clustering/k-means-clustering-tool/build-docker.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash

version=$(<VERSION)
docker build . -t polusai/k-means-clustering-tool:${version}
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
{
"name": "K-Means Clustering",
"version": "0.3.4-dev2",
"version": "0.3.5-dev0",
"title": "K-Means Clustering",
"description": "Cluster the data using K-Means.",
"author": "Jayapriya Nagarajan (jayapriya.nagarajan@nih.gov), Kelechi Nina Mezu (nina.mezu@nih.gov) Hamdah Shafqat Abbasi (hamdahshafqat.abbasi@nih.gov)",
"institution": "National Center for Advancing Translational Sciences, National Institutes of Health",
"repository": "https://github.com/polusai/polus-plugins",
"website": "https://ncats.nih.gov/preclinical/core/informatics",
"citation": "",
"containerId": "polusai/k-means-clustering-plugin:0.3.4-dev2",
"containerId": "polusai/k-means-clustering-tool:0.3.5-dev0",
"baseCommand": [
"python3",
"-m",
"polus.plugins.clustering.k_means"
"polus.images.clustering.k_means"
],
"inputs": [
{
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "polus-plugins-clustering-k-means"
version = "0.3.4-dev2"
name = "polus-images-clustering-k-means"
version = "0.3.5-dev0"
description = ""
authors = [
"Jayapriya Nagarajan <jayapriya.nagarajan@nih.gov>",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
"""K_means clustering."""

__version__ = "0.3.4-dev2"
__version__ = "0.3.5-dev0"
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@

import filepattern as fp
import typer
from polus.plugins.clustering.k_means import k_means as km
from polus.images.clustering.k_means import k_means as km

# Initialize the logger
logging.basicConfig(
format="%(asctime)s - %(name)-8s - %(levelname)-8s - %(message)s",
datefmt="%d-%b-%y %H:%M:%S",
)
logger = logging.getLogger("polus.plugins.clustering.k_means")
logger = logging.getLogger("polus.images.clustering.k_means")
logger.setLevel(os.environ.get("POLUS_LOG", logging.INFO))
POLUS_TAB_EXT = os.environ.get("POLUS_TAB_EXT", ".arrow")

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@
from sklearn.cluster import KMeans
from sklearn.metrics import calinski_harabasz_score, davies_bouldin_score


from polus.plugins.clustering.k_means.utils import Methods
from .utils import Methods

# Initialize the logger
logging.basicConfig(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
import filepattern as fp
import pytest
import vaex
from polus.plugins.clustering.k_means import k_means as km
from polus.plugins.clustering.k_means.__main__ import app
from polus.images.clustering.k_means import k_means as km
from polus.images.clustering.k_means.__main__ import app
from typer.testing import CliRunner

from .conftest import Generatedata
Expand Down
1 change: 0 additions & 1 deletion clustering/outlier-removal-plugin/VERSION

This file was deleted.

4 changes: 0 additions & 4 deletions clustering/outlier-removal-plugin/build-docker.sh

This file was deleted.

3 changes: 0 additions & 3 deletions clustering/outlier-removal-plugin/images/requirements.txt

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.5.0
current_version = 0.2.7-dev0
commit = True
tag = False
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\-(?P<release>[a-z]+)(?P<dev>\d+))?
Expand All @@ -20,10 +20,10 @@ values =
search = version = "{current_version}"
replace = version = "{new_version}"

[bumpversion:file:plugin.json]

[bumpversion:file:VERSION]

[bumpversion:file:README.md]

[bumpversion:file:src/polus/plugins/transforms/images/montage/__init__.py]
[bumpversion:file:plugin.json]

[bumpversion:file:src/polus/images/clustering/outlier_removal/__init__.py]
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
## [0.2.6-dev] - 2024-01-12
### Added
# [0.2.6-dev0] - 2024-01-12

## Added

- Pytests to test this plugin
- This plugin is now installable with pip.
- Added support for arrow file format in addition to csv

### Changed
## Changed

- Updated dependencies (bfio, filepattern, preadator) to latest
- Argparse package is replaced with Typer package for command line arguments
- Replaced docker base image with latest container image with pre-installed bfio
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@ COPY README.md ${EXEC_DIR}
RUN pip3 install ${EXEC_DIR} --no-cache


ENTRYPOINT ["python3", "-m", "polus.plugins.clustering.outlier_removal"]
ENTRYPOINT ["python3", "-m", "polus.images.clustering.outlier_removal"]
CMD ["--help"]
Original file line number Diff line number Diff line change
@@ -1,27 +1,31 @@
# Outlier removal
# Outlier removal (v0.2.7-dev0)

The outlier removal plugin removes the outliers from the data based on the method selected and outputs csv file. The output will have separate csv files for inliers and outliers. The input file should be in csv format.

The plugin support vaex supported input csv file that need outliers to be removed. The file should be in csv format. This is a required parameter for the plugin.

## Methods:
## Methods

Choose any one of the methods mentioned to remove outliers from the data.

### Isolation Forest

Ensemble-based unsupervised method for outlier detection. The algorithm isolates outliers instead of normal instances. It works based on the principle that outliers are few and different and hence, the outliers can be identified easier than the normal points. The score is calculated as the path length to isolate the observation. These two methods can be selected to detect outliers>
1. `IsolationForest`
Detect outliers globally that deviates significantly from the rest of the datapoints
2. `IForest`
Detect local outliers that are distinct when compared to those of its neighbors.

1. `IsolationForest` Detect outliers globally that deviates significantly from the rest of the datapoints
2. `IForest` Detect local outliers that are distinct when compared to those of its neighbors.


### Global

<img src="images/Global.PNG" width="500" height="500">

### Local

<img src="images/Local.PNG" width="500" height="500">

## Outputs:

Select the output file by passing value to `outputType`. User can select from following options `inlier`, `oulier` or `combined`. The combined file contains `anomaly` column which score each datapoint if it is inlier or outlier.

## Building
Expand Down
1 change: 1 addition & 0 deletions clustering/outlier-removal-tool/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0.2.7-dev0
4 changes: 4 additions & 0 deletions clustering/outlier-removal-tool/build-docker.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash

version=$(<VERSION)
docker build . -t polusai/outlier-removal-tool:${version}
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
{
"name": "Outlier Removal",
"version": "0.2.6-dev",
"version": "0.2.7-dev0",
"title": "Outlier Removal",
"description": "Remove outliers from the data.",
"author": "Jayapriya Nagarajan (github.com/Priyaaxle) and Hamdah Shafqat Abbasi (hamdahshafqat.abbasi@nih.gov)",
"institution": "National Center for Advancing Translational Sciences, National Institutes of Health",
"repository": "https://github.com/PolusAI/polus-plugins",
"website": "https://ncats.nih.gov/preclinical/core/informatics",
"citation": "",
"containerId": "polusai/outlier-removal-plugin:0.2.6-dev",
"containerId": "polusai/outlier-removal-tool:0.2.7-dev0",
"baseCommand": [
"python3",
"-m",
"polus.plugins.clustering.outlier_removal"
"polus.images.clustering.outlier_removal"
],
"inputs": {
"inpDir": {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "polus-plugins-clustering-outlier-removal"
version = "0.2.6-dev"
name = "polus-images-clustering-outlier-removal"
version = "0.2.7-dev0"
description = "Remove outliers from the data."
authors = [
"Jayapriya Nagarajan <jayapriya.nagarajan@axleinfo.com>",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
"""Outlier Removal Plugin."""
__version__ = "0.2.6-dev"

__version__ = "0.2.7-dev0"
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from typing import Optional

import filepattern as fp
import polus.plugins.clustering.outlier_removal.outlier_removal as rm
import polus.images.clustering.outlier_removal.outlier_removal as rm
import preadator
import typer

Expand All @@ -21,7 +21,7 @@
format="%(asctime)s - %(name)-8s - %(levelname)-8s - %(message)s",
datefmt="%d-%b-%y %H:%M:%S",
)
logger = logging.getLogger("polus.plugins.clustering.outlier_removal")
logger = logging.getLogger("polus.images.clustering.outlier_removal")


@app.command()
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""Test Command line Tool."""
from typer.testing import CliRunner
from polus.plugins.clustering.outlier_removal.__main__ import app
from polus.images.clustering.outlier_removal.__main__ import app
import shutil
from pathlib import Path

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

import filepattern as fp
import numpy as np
import polus.plugins.clustering.outlier_removal.outlier_removal as rm
import polus.images.clustering.outlier_removal.outlier_removal as rm
import vaex


Expand Down
27 changes: 27 additions & 0 deletions features/feature-segmentation-eval-tool/.bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
[bumpversion]
current_version = 0.2.6-dev0
commit = True
tag = False
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+)(\-(?P<release>[a-z]+)(?P<dev>\d+))?
serialize =
{major}.{minor}.{patch}-{release}{dev}
{major}.{minor}.{patch}

[bumpversion:part:release]
optional_value = _
first_value = dev
values =
dev
_

[bumpversion:part:dev]

[bumpversion:file:pyproject.toml]
search = version = "{current_version}"
replace = version = "{new_version}"

[bumpversion:file:plugin.json]

[bumpversion:file:VERSION]

[bumpversion:file:src/polus/images/features/feature_segmentation_eval/__init__.py]
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@ COPY src ${EXEC_DIR}/src

RUN pip3 install ${EXEC_DIR} --no-cache-dir

ENTRYPOINT ["python3", "-m", "polus.plugins.features.feature_segmentation_eval"]
ENTRYPOINT ["python3", "-m", "polus.images.features.feature_segmentation_eval"]
CMD ["--help"]
1 change: 1 addition & 0 deletions features/feature-segmentation-eval-tool/VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0.2.6-dev0
4 changes: 4 additions & 0 deletions features/feature-segmentation-eval-tool/build-docker.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#!/bin/bash

version=$(<VERSION)
docker build . -t polusai/feature-segmentation-eval-tool:${version}
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
{
"name": "Feature Segmentation Eval",
"version": "0.2.5-dev0",
"version": "0.2.6-dev0",
"title": "Feature Segmentation Eval",
"description": "Plugin to generate evaluation metrics for feature comparison of ground truth and predicted images.",
"author": "Vishakha Goyal (vishakha.goyal@nih.gov), Hamdah Shafqat Abbasi (hamdahshafqat.abbasi@nih.gov)",
"institution": "National Center for Advancing Translational Sciences, National Institutes of Health",
"repository": "https://github.com/labshare/polus-plugins",
"website": "https://ncats.nih.gov/preclinical/core/informatics",
"citation": "",
"containerId": "polusai/feature-segmentation-eval:0.2.5-dev0",
"containerId": "polusai/feature-segmentation-eval-tool:0.2.6-dev0",
"baseCommand": [
"python3",
"-m",
"polus.plugins.features.feature_segmentation_eval"
"polus.images.features.feature_segmentation_eval"
],
"inputs": [
{
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "polus-plugins-features-feature-segmentation-eval"
version = "0.2.5-dev0"
name = "polus-images-features-feature-segmentation-eval"
version = "0.2.6-dev0"
description = "Feature segmentation eval"
authors = [
"Vishakha Goyal <vishakha.goyal@nih.gov>",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Feature segmentation evaluation package."""
__version__ = "0.2.5-dev0"
__version__ = "0.2.6-dev0"
from . import feature_evaluation
from . import metrics
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,15 @@
from typing import Any

import filepattern as fp
import polus.plugins.features.feature_segmentation_eval.feature_evaluation as fs
import polus.images.features.feature_segmentation_eval.feature_evaluation as fs
import typer

logging.basicConfig(
format="%(asctime)s - %(name)-8s - %(levelname)-8s - %(message)s",
datefmt="%d-%b-%y %H:%M:%S",
)
POLUS_LOG = getattr(logging, environ.get("POLUS_LOG", "INFO"))
logger = logging.getLogger("polus.plugins.features.feature_segmentation_eval")
logger = logging.getLogger("polus.images.features.feature_segmentation_eval")
logger.setLevel(POLUS_LOG)
logging.getLogger("bfio").setLevel(POLUS_LOG)
# Set number of threads for scalability
Expand Down
Loading
Loading