Skip to content

Commit

Permalink
update tutorials/pretraining-Vietnamese-data-curation according to Sa…
Browse files Browse the repository at this point in the history
…rahyurick's comments

Signed-off-by: hoangphu7122002 <hoangphu7122002ai@gmail.com>
  • Loading branch information
hoangphu7122002 committed Oct 24, 2024
1 parent 5e666a9 commit f0cc43f
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -748,9 +748,9 @@
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from nemo_curator.modules import ExactDuplicates\n",
"from nemo_curator.datasets import DocumentDataset\n",
"import os\n",
"\n",
"# Define input and output paths\n",
"exact_dedup_input_dataset_dir = added_id_output_path\n",
Expand Down

0 comments on commit f0cc43f

Please sign in to comment.