Skip to content

Pull requests: NVIDIA/NeMo-Curator

New pull request New
23 Open 193 Closed
23 Open 193 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Semantic deduplication improvements
#327 opened Oct 25, 2024 by sarahyurick • Draft
1 of 2 tasks
Add codepath for computing buckets without int conversion
#326 opened Oct 25, 2024 by ayushdg • Draft
3 tasks
Add support for fine-tune classifier
#325 opened Oct 25, 2024 by VibhuJawa • Draft
Add nemo_curator.__version__, --version, and -v displays
#323 opened Oct 24, 2024 by sarahyurick Loading…
1
Dapt data curation fuzzy dedupe
#322 opened Oct 24, 2024 by ruchaa-apte Loading…
4
Add more comments for clearing Fuzzy Dedup Cache
#321 opened Oct 23, 2024 by praateekmahajan Loading…
3 tasks
1
Japanese support for get_word_splitter
#320 opened Oct 23, 2024 by sarahyurick Loading…
1
Skip reading files with incorrect extension
#318 opened Oct 22, 2024 by sarahyurick Loading…
1
4
Retiring text_bytes_aware_shuffle to use shuffle directly gpuci Run GPU CI/CD on PR
#316 opened Oct 21, 2024 by praateekmahajan • Draft
3 tasks
1
2
Make max_text_bytes_per_part configurable
#314 opened Oct 18, 2024 by sarahyurick Loading…
1
1
[WIP] MinHash improvement using minhash_permuted
#313 opened Oct 18, 2024 by praateekmahajan • Draft
3 tasks
[REVIEW] Speedup Connected Components
#302 opened Oct 15, 2024 by VibhuJawa Loading…
3 tasks done
25
add tutorials/pretraining-Vietnamese-data-curation
#300 opened Oct 14, 2024 by hoangphu7122002 Loading…
3 tasks
72
[DRAFT] Passing meta to map_partitions for read_data
#291 opened Oct 9, 2024 by praateekmahajan • Draft
3 tasks
1
[DRAFT] Trying dask_cudf's read_json / read_parquet
#285 opened Oct 8, 2024 by praateekmahajan • Draft
3 tasks
Added example notebook for translation with ct2 model. documentation Improvements or additions to documentation
#262 opened Sep 25, 2024 by uahmed93 • Draft
3 tasks
19
Added a translation pipeline for ctranslate2 inference
#245 opened Sep 16, 2024 by uahmed93 • Draft
3 tasks
Add support for parallel data curation
#193 opened Aug 8, 2024 by shuoyangd Loading…
3 tasks done
@shuoyangd
50
Fixed bug: changed to correct model name
#186 opened Aug 6, 2024 by ByteWrite Loading…
1 of 3 tasks
@ByteWrite
2
Add Multiple Model Quality Classification example documentation Improvements or additions to documentation
#173 opened Jul 30, 2024 by sarahyurick • Draft
1
@sarahyurick
3
Adding an example for executing NeMo modules using kubernetes Python … documentation Improvements or additions to documentation
#148 opened Jul 9, 2024 by dpadmanabhan03 Loading…
2 of 3 tasks
43
[WIP][Experiment] Use multi-threading for multi-file json reads
#94 opened May 31, 2024 by rjzamora • Draft
1
Fix #53 - Add batched files reading support to separate_by_metadata script
#54 opened May 6, 2024 by miguelusque Loading…
1
ProTip! Exclude everything labeled bug with -label:bug.