Skip to content

Efficient Distributed GPU Programming for Exascale, an SC/ISC Tutorial

License

Notifications You must be signed in to change notification settings

marcelloPuligheddu/tutorial-multi-gpu-ISC2023

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ISC23 Tutorial: Efficient Distributed GPU Programming for Exascale

DOI

Repository with talks and exercises of our Efficient GPU Programming for Exascale tutorial, to be held at ISC23.

Coordinates

  • Date: 21 May 2023
  • Occasion: ISC23 Tutorial
  • Tutors: Simon Garcia (SNL), Andreas Herten (JSC), Markus Hrywniak (NVIDIA), Jiri Kraus (NVIDIA), Lena Oden (Uni Hagen)

Setup

The tutorial is an interactive tutorial with introducing lectures and practical exercises to apply knowledge. The exercises have been derived from the Jacobi solver implementations available in NVIDIA/multi-gpu-programming-models.

Curriculum:

  1. Lecture: Tutorial Overview, Introduction to System + Onboarding Andreas
  2. Lecture: MPI-Distributed Computing with GPUs Simon
  3. Hands-on: Multi-GPU Parallelization
  4. Lecture: Performance / Debugging Tools Markus
  5. Lecture: Optimization Techniques for Multi-GPU Applications Markus
  6. Hands-on: Overlap Communication and Computation with MPI
  7. Lecture: Overview of NCCL and NVSHMEN in MPI Lena
  8. Hands-on: Using NCCL and NVSHMEM
  9. Lecture: Device-initiated Communication with NVSHMEM Lena
  10. Hands-on: Using Device-Initiated Communication with NVSHMEM
  11. Lecture: Conclusion and Outline of Advanced Topics Andreas

About

Efficient Distributed GPU Programming for Exascale, an SC/ISC Tutorial

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Cuda 44.9%
  • C++ 33.9%
  • Jupyter Notebook 12.4%
  • Makefile 7.0%
  • Shell 1.0%
  • TeX 0.6%
  • Python 0.2%