Skip to content

Commit

Permalink
CI: CUDA tests (#1729)
Browse files Browse the repository at this point in the history
  • Loading branch information
@casperdcl
casperdcl committed Mar 25, 2024
2 parents a5d51a5 + 82da339 commit 755801d
Show file tree
Hide file tree
Showing 14 changed files with 226 additions and 253 deletions.
94 changes: 39 additions & 55 deletions .github/workflows/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,70 +2,54 @@

There is a single github action file with multiple jobs, which builds both the conda package and documentation, and optionally publishes the documentation: [build](https://github.com/TomographicImaging/CIL/blob/master/.github/workflows/build.yml)

## Building the Conda Package: conda job
The jobs are:

This github action builds and tests the conda package, by using the [conda-package-publish-action](https://github.com/TomographicImaging/conda-package-publish-action)
- `test-cuda`
+ uses our self-hosted (STFC Cloud) CUDA-enabled runners to run GPU tests
+ `TESTS_FORCE_GPU=1 python -m unittest discover -v -k tigre -k TIGRE -k astra -k ASTRA -k gpu -k GPU ./Wrappers/Python/test`
- `test`
+ uses default (GitHub-hosted) runners to run tests on the min & max supported Python & NumPy versions
+ `python -m unittest discover -v ./Wrappers/Python/test`
- conda
+ uses `mambabuild` to build the conda package (saved as a build artifact named `cil-package`)
- docs
+ uses `docs/docs_environment.yml` plus `make -C docs` to build the documentation (saved as a build artifact named `DocumentationHTML`)
+ renders to the `gh-pages` branch on `master` (nightly) pushes or on tag (release) pushes
* this in turn is hosted at <https://tomographicimaging.github.io/CIL> as per <https://github.com/TomographicImaging/CIL/settings/pages>
- docker
+ builds a docker image from [`Dockerfile`](../../Dockerfile) (pushed to `ghcr.io/tomographicimaging/cil` as per [the CIL README section on Docker](../../README.md#docker))

When pushing to master or creating an [annotated tag](https://git-scm.com/book/en/v2/Git-Basics-Tagging), *all* variants are built and tested.
Details on some of these jobs are given below.

When opening or modifying a pull request to master, a single variant is built and tested. This variant is for linux with `python=3.9` and `numpy=1.22`.
## conda

The action does not publish to conda, instead this is done by jenkins. This is because github-actions do not have a GPU.
When opening or modifying a pull request to `master`, a single variant is built and tested. This variant is for linux with `python=3.9` and `numpy=1.22`.

It looks for conda-build dependencies in the channels listed [here](https://github.com/TomographicImaging/CIL/blob/master/.github/workflows/build.yml#L49). If you add any new dependencies, the appropriate channels need to be added to this line.
> [!NOTE]
> The action does not publish to conda, instead this is done by jenkins. We will eventually move from jenkins to conda-forge instead.
> When pushing to `master` or creating an [annotated tag](https://git-scm.com/book/en/v2/Git-Basics-Tagging), *all* variants are built and tested.
An artifact of the resulting tar.bz2 file is made available in the 'Summary' section of the action. It is called `cil-package`. This is used by the **docs** job. It can be found by going to the ‘Actions’ tab, and selecting the appropriate run of `.github/workflows/build.yml`, or by clicking on the tick on the action in the "All checks have passed/failed" section of a PR. When viewing the summary for the run of the action, there is an `Artifact` section at the bottom of the page. Clicking on `cil-package` allows you to download a zip folder containing the tar.bz2 file.
It looks for conda-build dependencies in the channels listed [here](https://github.com/TomographicImaging/CIL/blob/master/.github/workflows/build.yml#L118). If you add any new dependencies, the appropriate channels need to be added to this line.

## Building/Publishing Documentation: docs job
> [!TIP]
> The `conda` job builds the `*.tar.bz2` package and uploads it as an artifact called `cil-package`.
> It can be found by going to the "Actions" tab, and selecting the appropriate run of `.github/workflows/build.yml`, or by clicking on the tick on the action in the "All checks have passed/failed" section of a PR. When viewing the "Summary" for the run of the action, there is an "Artifact" section at the bottom of the page.
> Clicking on `cil-package` allows you to download a zip folder containing the `*.tar.bz2` file.
This github action builds and optionally publishes the documentation located in [docs/source](https://github.com/TomographicImaging/CIL/tree/master/docs/source). To do this it uses a forked version of the [build-sphinx-action](https://github.com/lauramurgatroyd/build-sphinx-action).

The [docs](https://github.com/TomographicImaging/CIL/blob/master/.github/workflows/build.yml#L59) job:

- creates a miniconda environment from [docs_environment.yml](https://github.com/TomographicImaging/CIL/blob/master/.github/workflows/docs/docs_environment.yml)
- installs cil into the miniconda environment, using the tar.bz2 artifact (cil-package) created in the **conda** job
- builds the documentation with sphinx
- uses upload-artifact to upload the html files: `HTMLDocumentation`, which can be downloaded to view
- pushes the html files to the `nightly` folder on the gh-pages branch

If opening or modifying a pull request to master, `docs` is run, but the final gh-pages step is skipped.
If pushing to master or tagging, the documentation is built *and* published to gh-pages.

### Viewing Built Documentation

The `docs` job builds the documentation and uploads it as an artifact, in a folder named `DocumentationHTML`.
This can be found by going to the ‘Actions’ tab, and selecting the appropriate run of `.github/workflows/build.yml`, or by clicking on the tick on the action in the "All checks have passed/failed" section of a PR.

When viewing the `Summary` for the run of the action, there is an `Artifact` section at the bottom of the page.
Clicking on `DocumentationHTML` allows you to download a zip folder containing the built html files. This allows you to preview the documentation site before it is published.

### Publication of the Documentation
## docs

The documentation is hosted on the [github site](https://tomographicimaging.github.io/CIL/) associated with the repository.
This is built from the [gh-pages branch](https://github.com/TomographicImaging/CIL/tree/gh-pages).

If you are an admin of the CIL repository you are able to see the settings for the site by going to `Settings->Pages`.

To publish the documentation, the publish job of the gh-action pushes the documentation changes to the `gh-pages` branch.
Any push to this branch automatically updates the github site.

### Initial Setup of the Docs Site & Action

To get the action to work I first had to:

1. [Create a gh-pages branch](https://gist.github.com/ramnathv/2227408) - note this only worked in bash, not windows command line.
2. [Set the source](https://github.com/TomographicImaging/CIL/settings/pages) for our github pages to be the gh-pages branch.

I followed the examples on the [sphinx build action page](https://github.com/marketplace/actions/sphinx-build), specifically this [example workflow](https://github.com/ammaraskar/sphinx-action-test/blob/master/.github/workflows/default.yml)

## Building/Pushing the Docker Image: docker job
This github action builds and optionally publishes the documentation located in [docs/source](https://github.com/TomographicImaging/CIL/tree/master/docs/source). To do this it uses a forked version of the [build-sphinx-action](https://github.com/lauramurgatroyd/build-sphinx-action).

This builds a docker image using the `Dockerfile` at the root of this repository.
The [docs](https://github.com/TomographicImaging/CIL/blob/master/.github/workflows/build.yml#L124) job:

The image is also pushed to `ghcr.io/tomographicimaging/cil:TAG` (https://github.com/TomographicImaging/CIL/pkgs/container/cil), where `TAG` is given by:
- creates a `miniconda` environment from [requirements-test.yml](https://github.com/TomographicImaging/CIL/blob/master/scripts/requirements-test.yml) and [docs_environment.yml](https://github.com/TomographicImaging/CIL/blob/master/docs/docs_environment.yml)
- `cmake` builds & installs CIL into the `miniconda` environment
- builds the HTML documentation with `sphinx`
- uploads a `DocumentationHTML` artifact (which can be downloaded to view locally for debugging)
- pushes the HTML documentation to the `gh-pages` branch
+ only if pushing to `master` or tagging (skipped if pushing to a branch or a PR)

git ref | docker tag(s)
--|--
`master` branch | `master`
`vM.m.p` tag | `M.m.p`, `M.m`, `latest`
anything else | not pushed (only built)
> [!TIP]
> The `docs` job builds the documentation and uploads it as an artifact called `DocumentationHTML`.
> It can be found by going to the "Actions" tab, and selecting the appropriate run of `.github/workflows/build.yml`, or by clicking on the tick on the action in the "All checks have passed/failed" section of a PR. When viewing the "Summary" for the run of the action, there is an "Artifact" section at the bottom of the page.
> Clicking on `DocumentationHTML` allows you to download a zip folder containing the built HTML files. This allows you to preview the documentation site before it is published.
100 changes: 82 additions & 18 deletions .github/workflows/build.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,21 +35,87 @@ on:
- 'NOTICE.txt'
- 'README.md'
jobs:
conda:
runs-on: ubuntu-20.04
test-cuda:
defaults: {run: {shell: 'bash -el {0}'}}
runs-on: [self-hosted, python, cuda]
strategy:
matrix:
python-version: [3.9]
numpy-version: [1.22]
steps:
- uses: actions/checkout@v4
with: {fetch-depth: 0, submodules: recursive}
- id: reqs
name: set requirements
run: |
envname="${GITHUB_REPOSITORY##*/}-${GITHUB_RUN_ID}.${GITHUB_RUN_NUMBER}"
echo "envname=$envname" >> $GITHUB_OUTPUT
sed -ri -e 's/^(name: ).*/\1$envname/' -e '/ python=/d' -e 's/(.* numpy=).*/\1${{ matrix.numpy-version }}/' scripts/requirements-test.yml
- uses: conda-incubator/setup-miniconda@v3
with:
fetch-depth: 0
- name: conda-build
uses: TomographicImaging/conda-package-publish-action@v2
python-version: ${{ matrix.python-version }}
environment-file: scripts/requirements-test.yml
activate-environment: ${{ steps.reqs.outputs.envname}}
run-post: false
- id: build
name: build
run: |
cmake -S . -B ./build -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCONDA_BUILD=ON -DCMAKE_INSTALL_PREFIX="$CONDA_PREFIX"
cmake --build ./build --target install
- name: test
run: TESTS_FORCE_GPU=1 python -m unittest discover -v -k tigre -k TIGRE -k astra -k ASTRA -k gpu -k GPU ./Wrappers/Python/test
- if: always()
name: Post Run conda-incubator/setup-miniconda@v3
run: |
conda deactivate
conda env remove -n "${{ steps.reqs.outputs.envname }}"
sed -i '/${{ steps.reqs.outputs.envname }}/d' ~/.profile
test:
defaults: {run: {shell: 'bash -el {0}'}}
runs-on: ubuntu-latest
strategy:
matrix:
include:
- python-version: 3.8
numpy-version: 1.21
- python-version: '3.10'
numpy-version: 1.24
steps:
- uses: actions/checkout@v4
with: {fetch-depth: 0, submodules: recursive}
- name: set requirements
run: sed -ri -e '/ python=/d' -e 's/(.* numpy=).*/\1${{ matrix.numpy-version }}/' scripts/requirements-test.yml
- uses: conda-incubator/setup-miniconda@v3
with:
python-version: ${{ matrix.python-version }}
environment-file: scripts/requirements-test.yml
activate-environment: cil_dev
- name: build
run: |
cmake -S . -B ./build -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCONDA_BUILD=ON -DCMAKE_INSTALL_PREFIX="$CONDA_PREFIX"
cmake --build ./build --target install
- name: test
run: python -m unittest discover -v ./Wrappers/Python/test
conda:
defaults: {run: {shell: 'bash -el {0}'}}
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.9]
numpy-version: [1.22]
steps:
- uses: actions/checkout@v4
with: {fetch-depth: 0, submodules: recursive}
- uses: conda-incubator/setup-miniconda@v3
with:
subDir: recipe
channels: -c conda-forge -c intel -c ccpi
convert_win: false
convert_osx: false
test_pyver: 3.9
test_npver: 1.22
python-version: ${{ matrix.python-version }}
mamba-version: "*"
channels: conda-forge
- name: conda build & test
working-directory: recipe
run: |
conda install boa
conda mambabuild . -c conda-forge -c intel -c ccpi --python=${{ matrix.python-version }} --numpy=${{ matrix.numpy-version }} --output-folder .
- name: Upload artifact of the conda package
uses: actions/upload-artifact@v4
with:
Expand All @@ -60,9 +126,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: recursive
with: {fetch-depth: 0, submodules: recursive}
- uses: conda-incubator/setup-miniconda@v3
with: {python-version: '3.10'}
- name: install dependencies
Expand All @@ -74,8 +138,8 @@ jobs:
conda list
- name: build cil
run: |
cmake -S . -B ../build -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCONDA_BUILD=ON -DCMAKE_INSTALL_PREFIX="$CONDA_PREFIX"
cmake --build ../build --target install
cmake -S . -B ./build -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCONDA_BUILD=ON -DCMAKE_INSTALL_PREFIX="$CONDA_PREFIX"
cmake --build ./build --target install
- name: checkout docs
uses: actions/checkout@v4
with:
Expand Down Expand Up @@ -113,8 +177,8 @@ jobs:
- uses: actions/checkout@v4
with:
fetch-depth: 0
ref: ${{ github.event.pull_request.head.sha || github.ref }} # fix SHA
submodules: recursive
ref: ${{ github.event.pull_request.head.sha || github.ref }} # fix SHA
- uses: jlumbroso/free-disk-space@v1.3.1
with:
docker-images: false
Expand Down Expand Up @@ -156,6 +220,6 @@ jobs:
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
pass:
needs: [conda, docs, docker]
needs: [test-cuda, test, conda, docs, docker]
runs-on: ubuntu-latest
steps: [{run: echo success}]
11 changes: 11 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,17 @@ git submodule update --init --recursive
docker build . -t ghcr.io/tomographicimaging/cil
```


### Testing

One installed, CIL functionality can be tested using the following command:

```sh
export TESTS_FORCE_GPU=1 # optional, makes GPU test failures noisy
python -m unittest discover -v ./Wrappers/Python/test
```


## Citing CIL


Expand Down
27 changes: 9 additions & 18 deletions Wrappers/Python/test/test_DataProcessor.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,8 @@
from cil.processors.cilacc_binner import Binner_IPP


@unittest.skipUnless(has_ipp, "Requires IPP libraries")
class TestBinner_cillacc(unittest.TestCase):

@unittest.skipUnless(has_ipp, "Requires IPP libraries")
def test_binning_cpp(self):

shape_in = [4,12,16,32]
Expand All @@ -73,8 +72,6 @@ def test_binning_cpp(self):
with self.assertRaises(ValueError):
binner_cpp = Binner_IPP(shape_in,shape_out,start_index,binning)


@unittest.skipUnless(has_ipp, "Requires IPP libraries")
def test_binning_cpp_2D_data(self):

data = dataexample.SIMULATED_SPHERE_VOLUME.get()
Expand Down Expand Up @@ -104,8 +101,6 @@ def test_binning_cpp_2D_data(self):

numpy.testing.assert_allclose(binned_by_hand,binned_arr,atol=1e-6)


@unittest.skipUnless(has_ipp, "Requires IPP libraries")
def test_binning_cpp_4D(self):

shape_in = [9,21,40,92]
Expand Down Expand Up @@ -137,8 +132,6 @@ def test_binning_cpp_4D(self):

numpy.testing.assert_allclose(binned_by_hand,binned_arr,atol=1e-6)


@unittest.skipUnless(has_ipp, "Requires IPP libraries")
def test_binning_cpp_2D(self):

shape_in = [1,1,3,3]
Expand Down Expand Up @@ -172,9 +165,7 @@ def test_binning_cpp_2D(self):


class TestBinner(unittest.TestCase):

def test_set_up_processor(self):

ig = ImageGeometry(20,22,23,0.1,0.2,0.3,0.4,0.5,0.6,channels=24)
data = ig.allocate('random')

Expand Down Expand Up @@ -666,7 +657,7 @@ def test_process_data_container(self):


@unittest.skipUnless(has_tigre and has_nvidia, "TIGRE GPU not installed")
def test_imagedata_full(self):
def test_imagedata_full_tigre(self):
"""
This test bins a reconstructed volume. It then uses that geometry as the reconstruction window and reconstructs again.
Expand Down Expand Up @@ -697,7 +688,7 @@ def test_imagedata_full(self):


@unittest.skipUnless(has_astra and has_nvidia, "ASTRA GPU not installed")
def test_aqdata_full(self):
def test_aqdata_full_astra(self):
"""
This test bins a sinogram. It then uses that geometry for the forward projection.
Expand Down Expand Up @@ -732,7 +723,7 @@ def test_aqdata_full(self):


@unittest.skipUnless(has_astra and has_nvidia, "ASTRA GPU not installed")
def test_aqdata_full_origin(self):
def test_aqdata_full_origin_astra(self):
"""
This test bins a sinogram. It then uses that geometry for the forward projection.
Expand Down Expand Up @@ -1200,7 +1191,7 @@ def test_slice_acquisition_data(self):


@unittest.skipUnless(has_tigre and has_nvidia, "TIGRE GPU not installed")
def test_imagedata_full(self):
def test_imagedata_full_tigre(self):
"""
This test slices a reconstructed volume. It then uses that geometry as the reconstruction window and reconstructs again.
Expand Down Expand Up @@ -1230,7 +1221,7 @@ def test_imagedata_full(self):


@unittest.skipUnless(has_astra and has_nvidia, "ASTRA GPU not installed")
def test_aqdata_full(self):
def test_aqdata_full_astra(self):
"""
This test slices a sinogram. It then uses that geometry for the forward projection.
Expand Down Expand Up @@ -1266,7 +1257,7 @@ def test_aqdata_full(self):


@unittest.skipUnless(has_astra and has_nvidia, "ASTRA GPU not installed")
def test_aqdata_full_origin(self):
def test_aqdata_full_origin_astra(self):
"""
This test slices a sinogram. It then uses that geometry for the forward projection.
Expand Down Expand Up @@ -2126,7 +2117,7 @@ def test_results_wrap(self):


@unittest.skipUnless(has_tigre and has_nvidia, "TIGRE GPU not installed")
def test_pad_ad_full(self):
def test_pad_ad_full_tigre(self):
"""
This test pads a acquisition data asymmetrically.
It then compares the FBP of the padded and unpadded data on the same ImageGeometry.
Expand Down Expand Up @@ -2168,7 +2159,7 @@ def test_pad_ad_full(self):


@unittest.skipUnless(has_astra and has_nvidia, "ASTRA GPU not installed")
def test_pad_id_full(self):
def test_pad_id_full_astra(self):
"""
This test pads an image data asymmetrically.
It then compares the forward projection of the padded and unpadded phantom on the same AcquisitionGeometry.
Expand Down
Loading

0 comments on commit 755801d

Please sign in to comment.