CMakeLists.txt Improvements for CUDA

[kylosus]

This PR bumps cmake version to 3.17 and replaces the deprecated find_package(CUDA) with FindCUDAToolkit, with a number of improvements to the compilation process:

• CUDA Include and Library directories are now handled automatically by cmake
• CUDA architecture handling is reworked: No more regex in CMakeLists.txt or manual -gencode string generation in python code.
• CUDA source files are now included directly in the targets: cmake handles proper compilation and linking of device code automatically.
• Similar modifications to OpenMP and Threads targets

[kylosus]

tests/cuda_highdim_kernel_test.py is erroring on master branch too, but the test passes because the dace program exits with 0. This code in this branch errors and causes a segfault in pytest, which is why the test is failing. I don't know how this is possible or how it was unearthed by the changes in this PR.

$ # git checkout master
$ pytest --tb=short -m "gpu" tests/cuda_highdim_kernel_test.py
============================================================================================== test session starts ==============================================================================================
platform linux -- Python 3.10.12, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/user/nvidia/dace-test
configfile: pytest.ini
collected 9 items / 6 deselected / 3 selected                                                         

tests/cuda_highdim_kernel_test.py ...                                                                                                                                                                     [100%]

=============================================================================================== warnings summary ================================================================================================
tests/cuda_highdim_kernel_test.py::test_gpu
  /home/user/nvidia/dace-test/venv/lib/python3.10/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <function CompiledSDFG.__del__ at 0x7f53aff3f1c0>
  
  Traceback (most recent call last):
    File "/home/user/nvidia/dace-test/dace/codegen/compiled_sdfg.py", line 402, in __del__
      self.finalize()
    File "/home/user/nvidia/dace-test/dace/codegen/compiled_sdfg.py", line 354, in finalize
      raise RuntimeError(
  RuntimeError: An error was detected after running "tests_cuda_highdim_kernel_test_highdim": invalid configuration argument. Consider enabling synchronous debugging mode (environment variable: DACE_compiler_cuda_syncdebug=1) to see where the issue originates from.
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================== 3 passed, 6 deselected, 1 warning in 19.23s ==================================================================================
ERROR launching kernel kernel_0_0_0: invalid configuration argument (9). Grid dimensions: (15, 6, 6); Block dimensions: (5, 0, 4).

$ python tests/cuda_highdim_kernel_test.py
High-dimensional GPU kernel test (12, 3, 14, 15, 1, 2, 3, 4, 5)
Difference: 0.0
High-dimensional GPU kernel test (12, 3, 14, 15, 1, 2, 3, 4, 5)
ERROR launching kernel kernel_0_0_0: invalid configuration argument (9). Grid dimensions: (15, 6, 6); Block dimensions: (5, 0, 4).
Exception ignored in: <function CompiledSDFG.__del__ at 0x7f259e54fd00>
Traceback (most recent call last):
  File "/home/user/nvidia/dace-test/dace/codegen/compiled_sdfg.py", line 402, in __del__
    self.finalize()
  File "/home/user/nvidia/dace-test/dace/codegen/compiled_sdfg.py", line 354, in finalize
    raise RuntimeError(
RuntimeError: An error was detected after running "highdim": invalid configuration argument. Consider enabling synchronous debugging mode (environment variable: DACE_compiler_cuda_syncdebug=1) to see where the issue originates from.
Difference: 0.0
WARNING: New access a[i:i + 2, j] already covered by a[i:i + 2, j:j + 2]

$ echo $?
0

[tbennun]

@kylosus thank you, I'll take a look

[kylosus]

I think it's because the old cmake links to static libcudart_static.a so the shared library loaded here never sees the error. This way handling cuda errors seems a little hacky, maybe __dace_runkernel_* and __program_{name}_internal functions should return the errors instead.

[tbennun]

maybe __dace_runkernel_* and __program_{name}_internal functions should return the errors instead.

This is something we considered, but cannot reliably implement in the code generator without significant effort.

[BenWeber42]

Hi, what's the status of this PR? Looks like this would be a nice change for DaCe?

[kylosus]

Hi, what's the status of this PR? Looks like this would be a nice change for DaCe?

@BenWeber42 It shoud be ready to merge, but the cmake can probably be improved further. I didn't touch parts unrelated to CUDA.

[BenWeber42]

Ok, thanks for the clarification. I think it's easiest to have this PR only be about CUDA related improvements to the CMakeLists.txt (I edited the title accordingly).

Could you maybe merge latest master into this branch? Since it hasn't been synced in quite a while.

[kylosus]

Seems fine

[BenWeber42]

There was a segfault in the gpu test. I restarted it...

[kylosus]

There was a segfault in the gpu test. I restarted it...

The failing test had been silently ignored until this PR. See the discussion above

[tbennun]

@BenWeber42 @kylosus I fixed this test in #1441. Turns out there were some invalid (empty) ranges in the maps of one of the tests

[BenWeber42]

Just approved the fix. Thanks again. That's great, then it looks like we should continue here as follows:

• Wait for Fix CUDA high-dimensional test #1441 to be merged
• Re-run tests (they should pass with Fix CUDA high-dimensional test #1441 merged)
• Find reviewer & merge

[tbennun]

@kylosus could you please update this PR to the latest master? I fixed the issue you were observing.