Skip to content

Commit

Permalink
updates for 5.5.0 (#26)
Browse files Browse the repository at this point in the history
  • Loading branch information
aaroncblack authored Jun 27, 2024
1 parent b7867c4 commit 5fd8c13
Show file tree
Hide file tree
Showing 120 changed files with 10,570 additions and 4,231 deletions.
38 changes: 1 addition & 37 deletions BUILDING.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,4 @@
UMT requires CMake to build. It is recommended that your are familiar with CMake, in order to enable/disable the supported build options for UMT. An example shell script 'build_and_run_umt.sh' is included in the UMT that will compile UMT, and its required libraries, on a typical Linux distribution. The Linux distribution must have an MPI installation available.

A list of common CMake options for UMT, and their default values, are:

Choose the type of build, options are: None Debug Release RelWithDebInfo MinSizeRel
**CMAKE_BUILD_TYPE:STRING=**

Link and enable the use of the optional CALIPER performance measurement library.
**ENABLE_CALIPER:BOOL=OFF**

Enable compiling CUDA code
**ENABLE_CUDA:BOOL=OFF**

Enables CMake's Find MPI support (Turn off when compiling with the mpi wrapper directly)
**ENABLE_FIND_MPI:BOOL=ON**

Enable OpenMP pragmas
**ENABLE_OPENMP:BOOL=OFF**

Enable OpenMP target offload pragmas
**ENABLE_OPENMP_OFFLOAD:BOOL=OFF**

Link and enable the use of the optional UMPIRE memory library. Umpire provides memory pools on both CPU and GPU devices, and CPU memory pools in page-locked memory.
**ENABLE_UMPIRE:BOOL=OFF**

Compiler OpenMP runtime implementation includes Fortran interface
**OPENMP_HAS_FORTRAN_INTERFACE:BOOL=OFF**

Compiler OpenMP implementation includes use_device_addr pragma. This is usually true for OpenMP 5.x implementations, but not for OpenMP 4.x.
**OPENMP_HAS_USE_DEVICE_ADDR:BOOL=OFF**

Use preprocessor macros that strictly conform to FPP.
**STRICT_FPP_MODE:BOOL=OFF**




UMT requires CMake to build. It is recommended that your are familiar with CMake, in order to enable/disable the supported build options for UMT. An example shell script is included in the UMT that will compile UMT, and its required libraries, on a typical Linux distribution. The Linux distribution must have an MPI installation available.

UMT also provides a Spack package and can be optionally built using that package manager. For more information on Spack see https://github.com/spack/spack.

Expand Down
24 changes: 0 additions & 24 deletions DEPENDENCIES.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,32 +2,8 @@ System libraries
-----------------------
UMT Requires MPI to be installed on your system to provide a C++ and Fortran compiler.

To build UMT with LLVM Clang/Flang, you must also use them to build MPI so that
Fortran mod files are compatible. For example, you might build OpenMPI as
follows:

```
$ CC=clang CXX=clang++ FC=flang-new \
CFLAGS=-O3 CXXFLAGS=-O3 FCFLAGS=-O3 \
../configure --prefix=$PWD/../install-flang \
--without-knem --without-ucx --enable-pretty-print-stacktrace \
--enable-orterun-prefix-by-default --enable-mpi1-compatibility
$ make -j
$ make install
```

Reference: <https://github.com/llvm/llvm-project/issues/56348>

Third party libraries
-----------------------
The code depends on several libraries.

Required libraries are:
- Conduit, a io interchange library
https://github.com/LLNL/conduit

Optional libraries:
- MFEM, a finite element methods library
https://github.com/mfem/mfem

MFEM requires the additional libraries METIS and HYPRE. See the MFEM github website for more information.
22 changes: 2 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,7 @@ UMT can run different test problems, depending on how it was configured and buil
Blueprint test problem
------------------------
UMT can dynamically generate an unstructured 2D or 3D mesh when built against Conduit
version 0.8.9 or later, or their develop branch as of 1/1/2024 or later. This mode is
selected using the -B command line argument.
version 0.8.9 or later. This mode is selected using the -B command line argument.
Each MPI rank in UMT will construct a mesh domain by repeating a Blueprint tile topology
into a larger, repeated pattern arranged as a square tiling of the tile topology.
By default, UMT will use the tile pattern that is built into Conduit, although other
Expand Down Expand Up @@ -71,7 +70,7 @@ high aspect ratio domains, then adjust the number of tiles in x,y,z using the `-
command line argument.

To run this problem:
1. Build UMT (MFEM is not needed)
1. Build UMT
2. Run the test driver in 3D. In the below example, the problem will run for 10 cycles. The `-d 10,10,10` argument will create a mesh domain with 10x10x10 = 1000 tiles, resulting in ~24000 zones/domain. Overall, there would be 8 times that number of zones due to running on 8 MPI ranks. The domains will be arranged in 2x2x2 layout. Run 'test_driver -h' for more info on the arguments.
```
srun -n 8 /path/to/install/bin/test_driver -c 10 -B -d 10,10,10
Expand All @@ -81,23 +80,6 @@ srun -n 8 /path/to/install/bin/test_driver -c 10 -B -d 10,10,10
srun -n 4 /path/to/install/bin/test_driver -c 10 -B -d 10,10,0
```

MFEM test problem
-------------------
UMT includes an unstructured mesh 3d test problem using a [MFEM](https://mfem.org/) mesh
This mesh can be refined using MFEM at run time to provide larger problems. This problem
requires UMT to have been built with MFEM support.

To run this problem:
1. Build UMT. This will produce a test_driver and makeUnstructuredBox executable.
2. Run the makeUnstructuredBox to produce the 3d test mesh.
```
srun -n1 /path/to/install/bin/makeUnstructuredBox
```
3. Run the test driver. In the below example the problem will run for 10 cycles and the mesh will be refined. Run 'test_driver -h' for more info on the arguments.
```
srun -n2 /path/to/install/bin/test_driver -i ./unstructBox3D.mesh -c 10 -r 1 -R 6
```

References
==============
* Nemanic, M K, and Nowak, P. "Radiation transport calculations on unstructured
Expand Down
62 changes: 15 additions & 47 deletions build_and_run_umt.sh
Original file line number Diff line number Diff line change
@@ -1,9 +1,4 @@
#!/bin/bash -xe
# This script will compile a basic Release build of UMT. Additional CMake options can be added to the command line args of this
# script, and they will be picked up and added to the UMT CMake command at the bottom of this script.
# For a list of supported CMake options, run 'ccmake /path/to/umt/src'.
# Do not copy this script out of the UMT repo directory, it assumes it is located next to the UMT source files in order to work.

#!/bin/sh -x
# If you have a UMT tarball, untar it. Otherwise, git clone it from github.
# git clone https://github.com/LLNL/UMT.git

Expand All @@ -17,26 +12,17 @@ CC=gcc
CXX=g++
FC=gfortran

# Set to 1 to optionally build UMT with UMPIRE support.
# For more information on UMPIRE CMake options, please see:
# https://umpire.readthedocs.io/en/develop/sphinx/advanced_configuration.html
USE_UMPIRE=0

FFLAGS=-fallow-argument-mismatch

# Intel example
#CC=icx
#CXX=icpx
#FC=ifx
# CC=icx
# CXX=icpx
# FC=ifx


# LLVM Clang/Flang build
#
# See DEPENDENCIES.md for a compatible MPI.
#
# So far this works only for single-threaded CPU execution.
#CC=clang
#CXX=clang++
#FC=flang-new
#FFLAGS=-flang-experimental-polymorphism
# This script will compile a basic Release build of UMT. Additional CMake options can be added to the command line args of this script, and they will be picked up and added to the UMT CMake command at the bottom of this script.
# For a list of supported CMake options, run 'ccmake /path/to/umt/src'.
# Do not copy this script out of the UMT repo directory, it assumes it is located next to the UMT source files in order to work.

# Get directory this script is located in. This is assumed to be the UMT repo location.
SOURCE="${BASH_SOURCE[0]}"
Expand All @@ -57,34 +43,16 @@ cd umt_workspace
git clone --recurse-submodules https://github.com/LLNL/conduit.git conduit -b v0.9.0
mkdir build_conduit
cd build_conduit
cmake ${PWD}/../conduit/src -DCMAKE_Fortran_FLAGS="${FFLAGS}" -DCMAKE_INSTALL_PREFIX=${INSTALL_PATH} -DCMAKE_C_COMPILER=${CC} -DCMAKE_CXX_COMPILER=${CXX} -DCMAKE_Fortran_COMPILER=${FC} -DMPI_CXX_COMPILER=mpicxx -DMPI_Fortran_COMPILER=mpifort -DBUILD_SHARED_LIBS=OFF -DENABLE_TESTS=OFF -DENABLE_EXAMPLES=OFF -DENABLE_DOCS=OFF -DENABLE_FORTRAN=ON -DENABLE_MPI=ON -DENABLE_PYTHON=OFF
cmake ${PWD}/../conduit/src -DCMAKE_INSTALL_PREFIX=${INSTALL_PATH} -DCMAKE_C_COMPILER=${CC} -DCMAKE_CXX_COMPILER=${CXX} -DCMAKE_Fortran_COMPILER=${FC} -DMPI_CXX_COMPILER=mpicxx -DMPI_Fortran_COMPILER=mpifort -DBUILD_SHARED_LIBS=OFF -DENABLE_TESTS=OFF -DENABLE_EXAMPLES=OFF -DENABLE_DOCS=OFF -DENABLE_FORTRAN=ON -DENABLE_MPI=ON -DENABLE_PYTHON=OFF
gmake -j install
cd ..

UMPIRE_CMAKE_ARGS=
UMPIRE_RUNLINE_ARGS=

if [ $USE_UMPIRE -eq 1 ]; then
echo "Enabling UMPIRE support"
# If building Umpire, enable it in the UMT CMake and provide the path to the installation.
UMPIRE_CMAKE_ARGS="-DENABLE_UMPIRE=TRUE -DUMPIRE_ROOT=${INSTALL_PATH}"
# If building Umpire, add the '-u 1' command line arg to the UMT test driver run line.
# This will tell it to use an Umpire CPU memory pool, or if a GPU run, to use a CPU pinned memory pool.
UMPIRE_RUNLINE_ARGS="-u 1"
git clone --recurse-submodules https://github.com/LLNL/Umpire.git -b v2023.06.0

mkdir build_umpire
cd build_umpire
cmake ${PWD}/../Umpire -DCMAKE_INSTALL_PREFIX=${INSTALL_PATH} -DCMAKE_C_COMPILER=${CC} -DCMAKE_CXX_COMPILER=${CXX} -DCMAKE_Fortran_COMPILER=${FC} -DMPI_CXX_COMPILER=mpicxx -DMPI_Fortran_COMPILER=mpifort -DBUILD_SHARED_LIBS=OFF -DENABLE_TESTS=OFF -DENABLE_EXAMPLES=OFF -DENABLE_DOCS=OFF -DENABLE_FORTRAN=ON -DENABLE_MPI=ON
gmake -j install
cd ..
fi

# Run CMake on UMT, compile, and install.
cmake ${UMT_REPO_PATH}/src -DCMAKE_Fortran_FLAGS=${FFLAGS} -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=${CXX} -DCMAKE_Fortran_COMPILER=${FC} -DCMAKE_INSTALL_PREFIX=${INSTALL_PATH} -DCONDUIT_ROOT=${INSTALL_PATH} ${UMPIRE_CMAKE_ARGS} $1
cmake ${UMT_REPO_PATH}/src -DCMAKE_Fortran_FLAGS=${FFLAGS} -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=${CXX} -DCMAKE_Fortran_COMPILER=${FC} -DCMAKE_INSTALL_PREFIX=${INSTALL_PATH} -DCONDUIT_ROOT=${INSTALL_PATH} $1
gmake -j install
cd ..

# Run two smoke tests to verify executable, one on 2D 8x8 tiles mesh and one on 3D 4x4x4 tiles mesh.
srun -n 8 ${INSTALL_PATH}/bin/test_driver -c 10 -B local -d 8,8,0 --benchmark_problem 2 ${UMPIRE_RUNLINE_ARG}
srun -n 8 ${INSTALL_PATH}/bin/test_driver -c 10 -B local -d 4,4,4 --benchmark_problem 2 ${UMPIRE_RUNLINE_ARG}
srun -n 8 ${INSTALL_PATH}/bin/test_driver -c 10 -B local -d 8,8,0 --benchmark_problem 2
srun -n 8 ${INSTALL_PATH}/bin/test_driver -c 10 -B local -d 4,4,4 --benchmark_problem 2

# Test UMT on SSP1 unstructured 3d mesh problem on two mpi ranks. Refine the mesh via -r and -R arguments.
2 changes: 1 addition & 1 deletion host-configs/example.gnu.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ set(CMAKE_Fortran_FLAGS "-ffree-line-length-none" CACHE PATH "")
set(ENABLE_OPENMP ON CACHE BOOL "")
set(ENABLE_OPENMP_OFFLOAD OFF CACHE BOOL "")

set(CONDUIT_ROOT ${TPL_ROOT}/conduit/develop CACHE PATH "")
set(CONDUIT_ROOT ${TPL_ROOT}/conduit/0.8.2 CACHE PATH "")
14 changes: 0 additions & 14 deletions spack/umt/package.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ class Umt(CachedCMakePackage, CudaPackage):
variant("openmp", default=False, description="Enable OpenMP support")
variant("openmp_offload", default=False, description="Enable OpenMP target offload support")
variant("caliper", default=False, description="Enable Caliper performance timers")
variant("mfem", default=False, description="Enable support for reading MFEM meshes")
variant("umpire", default=False, description="Enable use of Umpire memory library")
variant("shared", default=False, description="Enable shared libraries")
variant("silo", default=False, description="Enable silo I/O support")
Expand Down Expand Up @@ -196,19 +195,6 @@ def initconfig_package_entries(self):

entries.append(cmake_cache_path("CONDUIT_ROOT", spec["conduit"].prefix))

if "+mfem" in spec:
entries.append(cmake_cache_option("ENABLE_MFEM", True))
entries.append(cmake_cache_path("MFEM_ROOT", spec["mfem"].prefix))
if "hypre" in spec:
entries.append(cmake_cache_path("HYPRE_ROOT", spec["hypre"].prefix))
if "metis" in spec:
entries.append(cmake_cache_path("METIS_ROOT", spec["metis"].prefix))
if ("+zlib" in spec["mfem"]):
found_zlib_dependency = True

else:
entries.append(cmake_cache_option("ENABLE_MFEM", False))

if "+caliper" in spec:
entries.append(cmake_cache_option("ENABLE_CALIPER", True))
entries.append(cmake_cache_path("CALIPER_ROOT", spec["caliper"].prefix))
Expand Down
Loading

0 comments on commit 5fd8c13

Please sign in to comment.