-
Notifications
You must be signed in to change notification settings - Fork 75
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Add partial image implementation Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Refactor requirements Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix bugs Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Change from_map to map_partitions Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add super constructor Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add kwargs for load_object_on_worker Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Get proper epoch size Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Complete embedding creation loop Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Change devices Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add device Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Refactor embedding creation and add classifier Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix bugs in classifiers Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Refactor model names Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add model name Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix classifier bugs Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Allow postprocessing for classifiers Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix name and add print Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix variable name Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add NSFW Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Update init for import Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix embedding size Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add fused classifiers Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix missing index Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Update metdata for fused classifiers Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add export to webdataset Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix missing id col Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Sort embeddings by id Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add timm Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Update init file Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add autocast to timm Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Update requirements and transform Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add additional interpolation support Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix transform normalization Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Remove open_clip Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add index path support to wds Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Address Vibhu's feedback Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add import guard for image dataset Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Change default device Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Remove commented code Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Remove device id Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix index issue Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add docstrings and standardize variable names Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add image curation tutorial Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add initial image docs Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Remove tutorial Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add dataset docs Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add embedder documentation Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Revert embedding column name change Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Update user guide for images Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Update README Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Update README with RAPIDS nightly instructions Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix formatting issues in image documentation Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Remove extra newline in README Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Address most of Sarah's feedback Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Add section summary Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix errors and REWORD GPU bullets in README Signed-off-by: Ryan Wolf <rywolf@nvidia.com> * Fix how table of contents displays with new sections Signed-off-by: Ryan Wolf <rywolf@nvidia.com> --------- Signed-off-by: Ryan Wolf <rywolf@nvidia.com>
- Loading branch information
Showing
25 changed files
with
1,002 additions
and
131 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
====================================== | ||
Classifiers | ||
====================================== | ||
|
||
------------------------------ | ||
Base Class | ||
------------------------------ | ||
|
||
.. autoclass:: nemo_curator.image.classifiers.ImageClassifier | ||
:members: | ||
|
||
|
||
------------------------------ | ||
Image Classifiers | ||
------------------------------ | ||
|
||
.. autoclass:: nemo_curator.image.classifiers.AestheticClassifier | ||
:members: | ||
|
||
.. autoclass:: nemo_curator.image.classifiers.NsfwClassifier | ||
:members: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
====================================== | ||
Embedders | ||
====================================== | ||
|
||
------------------------------ | ||
Base Class | ||
------------------------------ | ||
|
||
.. autoclass:: nemo_curator.image.embedders.ImageEmbedder | ||
:members: | ||
|
||
|
||
------------------------------ | ||
Timm | ||
------------------------------ | ||
|
||
.. autoclass:: nemo_curator.image.embedders.TimmImageEmbedder | ||
:members: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
====================================== | ||
Image Curation | ||
====================================== | ||
|
||
.. toctree:: | ||
:maxdepth: 4 | ||
:titlesonly: | ||
|
||
embedders.rst | ||
classifiers.rst |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -18,4 +18,5 @@ API Reference | |
decontamination.rst | ||
services.rst | ||
synthetic.rst | ||
image/index.rst | ||
misc.rst |
File renamed without changes
File renamed without changes
File renamed without changes
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.