WRF

WRF ported on NCI machine: Gadi

Installing

You can install WRF on any filesystem on Gadi: /home, /g/data or /scratch. Keep in mind /scratch is purged regularly. It isn't a problem if you are using WRF without modifications (easy to reinstall from here) but you would need to save any code modifications on a different filesystem (or Github etc.) To use on Gadi:

• from version 4.4 onwards, run:

     git clone -b <VERSION> --recursive https://github.com/coecms/WRF.git

or

    git clone -b <VERSION> --recursive git@github.com:coecms/WRF.git      

where version is the WRF version you want to use.

• before version 4.4, run:

     git clone -b <VERSION> https://github.com/coecms/WRF.git

or

    git clone -b <VERSION> git@github.com:coecms/WRF.git      

where version is the WRF version you want to use.

Versions currently available:

• V4.6.0 (with updated WPS V4.6.0)
• V4.5.2
• V4.5.1
• V4.5 (with updated WPS V4.5)
• V4.4.3 (identical to V4.4.2)
• V4.4.2
• V4.4.1
• V4.4 (with updated WPS)
• V4.3 (with updated WPS)
• V4.2.2
• V4.2.1
• V4.2
• V4.1.3
• V4.1.2
• V4.1.1
• V4.0.2
• V3.9.1.1
• V3.9
• V3.7.1
• V3.6.1
• V3.6

Building WRF (ARW)

Go to the WRF/ subdirectory, run:

cd WRF/WRF/

There is a run_compile script to help you configure and compile WRF. This script by default will compile the WRFV3 software for a real experiment, with some speed optimisations for the normal, express. normalbw and expressbw queues and with basic nesting. There is a range of command-line arguments accepted by the script. These are:

• the command-line arguments accepted by the configure script
• an option to clean everything prior to compilation
• the choice of compilation flags
• the choice of nesting type for the configure script
• the compilation option for the compile script.

You can access a full list and description by running:

./run_compile -h

To configure and compile WRF with the default options, you simply run:

./run_compile

If you want more aggressive optimisation options on Cascade Lake nodes, please refer to the help of run_compile for the version you are using.

The configure step will run interactively with the output on your screen, then the compilation step will be submitted to the express queue automatically.

After a successful build, one should see the following executable files for WRF (the ARW core):

[abc123@gadi WRF]$ ls -l main/*.exe

-rwx------ 1 abc123 wrf 35559897 Aug 22 17:14 main/ndown.exe
-rwx------ 1 abc123 wrf 35471615 Aug 22 17:14 main/real.exe
-rwx------ 1 abc123 wrf 35081246 Aug 22 17:14 main/tc.exe
-rwx------ 1 abc123 wrf 38763058 Aug 22 17:13 main/wrf.exe

Tip: if you are only running an idealised case and you are getting tired of specifying the option on the command-line, you can:

1. change the default value of compile_case at the start of the run_compile script OR
2. create an alias for the command in your .bashrc or .cshrc file (it is a good idea to change the name of the command for the alias so it is clear it is an alias if you need help later on): bash syntax:

alias my_compile='run_compile --compile_case <replace with the compile option here>'

csh/tcsh syntax:

alias my_compile 'run_compile --compile_case <replace with the compile option here>'

Building WPS

NOTE: because of current unavailability of some software on Gadi, not all executables are built. This will be solved when the software needed are made available.

Go to the WPS/ subdirectory, run:

cd WRF/WPS/

There is another run_compile script to help with configuring and compiling WPS. You can again choose the configure input from the command-line option as well as to clean a previous compilation of WPS. The default is to compile with GRIB2 support and a distributed memory code. See the inline help for run_compile for a full explanation:

./run_compile -h

To configure and compile WPS with the default options, you simply run:

./run_compile

A file configure.wps should be created after configuring has completed.

After a successful build, one should see the following executable files: (Note: the util/ executables are not yet ported to Gadi)

[abc123@gadi WPS]$ ls -l *.exe

lrwxrwxrwx 1 abc123 wrf 23 Aug 23 10:19 geogrid.exe -> geogrid/src/geogrid.exe
lrwxrwxrwx 1 abc123 wrf 23 Aug 23 10:20 metgrid.exe -> metgrid/src/metgrid.exe
lrwxrwxrwx 1 abc123 wrf 21 Aug 23 10:20 ungrib.exe -> ungrib/src/ungrib.exe

[abc123@gadi WPS]$ ls -l */src/*.exe

-rwx------ 1 abc123 wrf 3412782 Aug 23 10:19 geogrid/src/geogrid.exe
-rwx------ 1 abc123 wrf 3206974 Aug 23 10:20 metgrid/src/metgrid.exe
-rwx------ 1 abc123 wrf 1209622 Aug 23 10:20 ungrib/src/g1print.exe
-rwx------ 1 abc123 wrf 1407507 Aug 23 10:20 ungrib/src/g2print.exe
-rwx------ 1 abc123 wrf 2169010 Aug 23 10:20 ungrib/src/ungrib.exe
-rwx------ 1 abc123 wrf 1184660 Aug 23 10:20 util/src/avg_tsfc.exe
-rwx------ 1 abc123 wrf 1250303 Aug 23 10:20 util/src/calc_ecmwf_p.exe
-rwx------ 1 abc123 wrf 1217842 Aug 23 10:20 util/src/height_ukmo.exe
-rwx------ 1 abc123 wrf 953821 Aug 23 10:20 util/src/int2nc.exe
-rwx------ 1 abc123 wrf 1118699 Aug 23 10:20 util/src/mod_levs.exe
-rwx------ 1 abc123 wrf 3704627 Aug 23 10:20 util/src/plotfmt.exe
-rwx------ 1 abc123 wrf 3409302 Aug 23 10:20 util/src/plotgrids.exe
-rwx------ 1 abc123 wrf 916613 Aug 23 10:20 util/src/rd_intermediate.exe

Note that there are some warnings in the build output like "warning: overriding commands for target '.c.o'." These are harmless and can be ignored.

The October 2016 tutorial case

The data for the October 2016 WRF tutorial case can now be found under /g/data/sx70/data/SingleDomain_data/matthew.

Tests run

Compilation

• WRF compilation with dm+sm
• WPS compilation with distributed memory with dm+sm compilation of WRF

Tests simulations

Tests simulations have been done on Jenkins. The tests were updated from version 4.3. From version 4.3, the tests were:

• First tutorial case (Oct 16) with 1 nest
• First tutorial case (Oct 16) with 2 nests
• First tutorial case (Oct 16) with additional diagnostics turned on
• First tutorial case (Oct 16) with restart

For previous versions, the simulations were:

• First tutorial case (Jan 00) with 1 nest
• First tutorial case (Jan 00) with 2 nests
• First tutorial case (Jan 00) with additional diagnostics turned on.
• First tutorial case (Jan 00) with quiting I/O.
• First tutorial case (Jan 00) with restart

WRF outputs for these tests can be found under /g/data/sx70/data/KGO/. The inputs for the tests can be found in https://github.com/coecms/wrf-testing