Skip to content

Commit

Permalink
add checkpoint and resume title in docs (#7050)
Browse files Browse the repository at this point in the history
  • Loading branch information
lhoestq authored Jul 15, 2024
1 parent bf6f41e commit 87f4c20
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 6 deletions.
4 changes: 2 additions & 2 deletions docs/source/use_with_jax.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ True
Note that if the `device` argument is not provided to `with_format` then it will use the default
device which is `jax.devices()[0]`.

## N-dimensional arrays
### N-dimensional arrays

If your dataset consists of N-dimensional arrays, you will see that by default they are considered as the same tensor if the shape is fixed:

Expand Down Expand Up @@ -120,7 +120,7 @@ To avoid this, you must explicitly use the [`Array`] feature type and specify th
[7, 8]]], dtype=int32)}
```

## Other feature types
### Other feature types

[`ClassLabel`] data is properly converted to arrays:

Expand Down
6 changes: 4 additions & 2 deletions docs/source/use_with_pytorch.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ To load the data as tensors on a GPU, specify the `device` argument:
{'data': tensor([1, 2], device='cuda:0')}
```

## N-dimensional arrays
### N-dimensional arrays

If your dataset consists of N-dimensional arrays, you will see that by default they are considered as the same tensor if the shape is fixed:

Expand Down Expand Up @@ -82,7 +82,7 @@ To avoid this, you must explicitly use the [`Array`] feature type and specify th
```


## Other feature types
### Other feature types

[`ClassLabel`] data are properly converted to tensors:

Expand Down Expand Up @@ -223,6 +223,8 @@ If the dataset is split in several shards (i.e. if the dataset consists of multi

In this case each worker is given a subset of the list of shards to stream from.

### Checkpoint and resume

If you need a DataLoader that you can checkpoint and resume in the middle of training, you can use the `StatefulDataLoader` from [torchdata](https://github.com/pytorch/data):

```py
Expand Down
4 changes: 2 additions & 2 deletions docs/source/use_with_tensorflow.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ array([[1, 2],
[3, 4]])>}
```

## N-dimensional arrays
### N-dimensional arrays

If your dataset consists of N-dimensional arrays, you will see that by default they are considered as the same tensor if the shape is fixed:

Expand Down Expand Up @@ -88,7 +88,7 @@ To avoid this, you must explicitly use the [`Array`] feature type and specify th
```


## Other feature types
### Other feature types

[`ClassLabel`] data are properly converted to tensors:

Expand Down

0 comments on commit 87f4c20

Please sign in to comment.