Skip to content

Developer Guide

Jump to bottom
Ali.Abdolali edited this page May 7, 2021 · 65 revisions

Development of WAVEWATCH III® (henceforth WW3) is undertaken to add features to the existing framework, including new science and technological advancements, improve performance, fix bugs etc. WW3 is a community model, and thus requires that a collaborative model be established with clear rules and coding ethics that are strictly followed by all developers. For that purpose, current WW3 development will adopt the GitFlow model.

This guide will take you through the main steps and practical actions that should be undertaken as part of this development model. If you are interested in further details, our page WW3 Gitflow provides more information. For information to clone, build and run WW3, see the Quick Start guide.

WW3 GitFlow Rationale

GitFlow is a branching style where there are two main branches, master and develop. The master branch contains stable, tested code that is ready for production type use, while develop is a faster moving branch where recent developments are reintegrated too. There are two other types of branches, feature and bugfix. These two types of branches are created from the develop branch for adding features or bug fixes respectively before being reintegrated to the develop branch.

WW3 has one authoritative repository part of the NOAA-EMC organization. This repository is maintained by NCEP/EMC code managers and will be used to produce public releases and is the central reference to all copies of the code. In addition to the authoritative repository, there are trusted institutional repositories/forks. A developer may choose to create a fork from the authoritative repo or from one of the trusted institutional repos.

All development will be made by forks out of one of these trusted institutional repositories. The develop branch is the primary source of code that is currently being updated. All new feature branches are created from there, and completed development threads target updates to that branch via pull request.

Updates to the master branch will be done by the authoritative fork. Updates to the develop branch should be done via a pull request to the authoritative or trusted institutional forks.

To contribute to WW3, developers will fork the authoritative repo or one of the trusted institutional repos using their own GitHub accounts. In the developer's fork, new developments should be made in a feature or bug fix branch created from the develop branch. Developers should make an effort to communicate with the code manager from the repository associated with the developer's fork to indicate what developments are being made.

Once a feature branch has been tested and is mature to become a candidate for reintegration to the main repo, the developer sends a pull request to the authoritative or institutional forks to start the process of reintegration.

Development Workflow

The "fork and branch" workflow we are adopting is explained in more detail in this Scott's Weblog post. The main steps are:

  1. Create an issue associated to the new development.
  2. Fork the WW3 GitHub repository.
  3. Clone the forked repository to your local system.
  4. Add a Git remote for the original repository.
  5. Create a feature branch, make changes, commit and push changes to your GitHub fork.
  6. Communicate with the source repository code manager to indicate what is being developed.
  7. Open a pull request from the new branch to the original repo.
  8. Clean up after your pull request is merged.

1.Create an issue associated to the new development.

  • Include the issue in the commit message.

  • Include the list of regression test(s) with expected changes due to this development.

  • Include the detail of the new/modified regression test to check this development.

  • Update the issue with relevant details if needed.

2. Fork the WW3 GitHub Repo

  • Create and account on github.com,
  • On github.com choose one of the WW3 trusted repositories
  • Locate and click the fork icon on the upper right-hand corner of the WW3 GitHub page.

This creates a full copy of the WW3 repository under your user account.

3. Clone the Forked Repo Locally

You can now navigate to your GitHub dashboard and choose the newly-created fork. To start your development work, you need to first create a local copy on your machine of your fork. To do that, you must git clone your fork locally. First, find the right address for the fork on your GitHub fork page clicking the "Clone or Download" dropdown menu. Then, on your local box:

git clone https://github.com/<username>/WW3.git

If you are using ssh:

git clone ssh://git@github.com/<username>/WW3.git

4. Add Remotes and update you fork/branch with authoritative/trusted repos

Remotes are GitHub repositories that are linked to your local copy. A first remote origin is automatically created when you clone a repo. In this case, that links your local copy to the fork you created in your GitHub account. You can keep your local copy synced with your fork and vice-versa by simply using commands such as git pull to sync the local copy with the fork, or git push the way around. We will also want to keep the local copy and your fork synced with changes made in the repository your forked from.

That is possible by creating a remote link within your local copy that points to the upstream repo:

git remote add upstream https://github.com/NOAA-EMC/WW3

or

git remote add upstream https://github.com/<TrustedInstitution>/WW3

Sync with the main repo

Syncing your local copy to the upstream remote is done within your local copy by using the commands:

  • Change to the active development branch
    git checkout develop

  • Copy over changes to upstream repo
    git fetch upstream

  • Merge changes to your local repo
    git merge upstream/develop

  • Push changes to your GitHub fork
    git push

Likewise, when the master changes, you will want to sync your fork accordingly.

Using git push to upstream repos will fail, and should not be attempted.

5. Creating a feature Branch for Development Work

Adding new or modifying existing features of WW3 should be done in a feature branch. This involves the development cycle steps below.

Before you get started, please note some rules of engagement here:

  • Send the managers of the repo you forked from an email indicating what you are planning to develop,
  • Create one feature branch for every single feature or single bugfix you will be working on. Do not bundle all features or bugfixes in a single branch,
    • This creates a situation where undeveloped features will get in the way of reintegrating mature work,
    • Pull requests with more than one feature per branch will be returned.

The development cycle steps:

  • Create a new feature branch from the develop branch git checkout -b <branch name> develop
  • Make changes to the files,
  • Test your changes using the regtests
    • To ensure regression tests work, you need to download large files that are not part of the WW3 GitHub repo,
    • In your local copy from the top level directory, run the script:
      sh model/bin/ww3_from_ftp.sh
    • Make note of the added files and ensure they remain untracked (eg, do not use git add for these files),
  • Frequently commit your changes to the branch.
    • Check changes made
      git status
    • Add files indicated by status
      git add <some files>
    • Commit changes locally,
      • Use a commit message in the form "branchname>: [description of changes]":
        git commit -m "<branch name>: [description of changes and files affected]"
  • Push changes to your GitHub fork
    git push origin <branch name>

7. Reintegrate to Main Repo: Open a Pull Request

Pull requests are proposed changes to a repository submitted by a user and accepted or rejected by the main repo code managers. This should happen when you made changes in one of your feature branches that are mature and could become part of the WW3 package. Before you submit a pull request, make sure to verify if your changes satisfy our checklist for a developer submitting pull requests. When you get to that point, follow the guidelines well-established in GitHub:

  • Using your browser, navigate to the original repository you created your fork from.

    • To the right of the Branch menu, click New pull request.

  • On the Compare page, click the highlighted link compare across forks.

  • Confirm that the base fork is the repository you'd like to merge changes into.

    • Use the base branch drop-down menu to select the branch of the upstream repository you'd like to merge changes into.

  • Use the head fork drop-down menu to select your fork,

    • then use the compare branch drop-down menu to select the branch you made your changes in.

  • Type a title and description for your pull request.

  • If you do not want to allow anyone with push access to the upstream repository to make changes to your PR, unselect Allow edits from maintainers.

  • To create a pull request that is ready for review, click Create Pull Request.

    • To create a draft pull request, use the drop-down and select Create Draft Pull Request, then click Draft Pull Request.

For more information about draft pull requests, see About pull requests"

8. Clean up after your pull request is merged.

Depending on if the code managers accept your new features/changes into the DEVELOP branch (note it will first go into a feature branch on the fork), you may either restart the development cycle, or have completed your work on that feature. If the work is complete and your changes are reintegrated into the develop branch (not just a branch on the fork), then its time to clean up!

  1. Update your local clone from the upstream develop branch (see Sync with upstream remote).

  2. The changes are already in the main repo's develop branch, its time to delete the feature branch locally,

git branch -d <branch name>

  1. Update the master branch in your forked repository,

git push origin master

  1. Push the deletion of the feature branch to your GitHub repo,

git push --delete origin <branch name>

Contact WW3 Code Managers if you have any questions or suggestions that may improve our collaboration.

Regression Testing in WAVEWATCH III®

A series of regression tests are prepared to check the WW3 with different options and physics. Developers are encouraged to check their implementations with the regression tests to ensure the code compilation, consistency of model output and data reproducibility. Regression tests are enforced when new master or develop branches are created, and this typically involves running a complete set including all available regression tests with different configurations (see “The Matrix” section below). If you add new features, it is also encouraged that you add a regtest for the new feature, as outlined in the checklist for adding or modifying a regtest.

The following regression tests are available under retests directory:

Test Label Description
ww3_tp1.1 1D propagation around the world along the equator (no land).
ww3_tp1.2 1D propagation, along meridian (no land).
ww3_tp1.3 1D propagation, shoaling test.
ww3_tp1.4 1D propagation, spectral refraction (x).
ww3_tp1.5 1D propagation, spectral refraction (y).
ww3_tp1.6 1D propagation, wave blocking by current.
ww3_tp1.7 1D propagation, IG wave generation.
ww3_tp1.8 1D propagation, wave breaking on a beach.
ww3_tp1.9 1D propagation, Beji and Battjes (1993) barred flume case.
ww3_tp1.10 1D propagation, Bottom scattering beach.
ww3_tp1.11 1D propagation, Test for spectral interpolation in ww3_bounc.
ww3_tp2.1 2D propagation under angle with grid.
ww3_tp2.2 2D propagation over half the globe without land (with directional spread).
ww3_tp2.3 2D propagation, GSE test.
ww3_tp2.4 2D propagation, East Pacific curvilinear grid test.
ww3_tp2.5 2D propagation, Arctic Grid, curvilinear grid test.
ww3_tp2.6 2D propagation, Limon Harbor unstructured grid test.
ww3_tp2.7 Reflection on a 2D unstructured grid.
ww3_tp2.8 Tidal constituents on a 2D regular grid.
ww3_tp2.9 Tests for obstruction grids.
ww3_tp2.10 Tests for SMC grid.
ww3_tp2.11 Tests for rotated grid.
ww3_tp2.12 Test for system tracking.
ww3_tp2.13 Test for propagation under angle with grid (tripole).
ww3_tp2.14 Test for toy-model using OASIS coupler.
ww3_tp2.15 Test for space-time extremes parameters.
ww3_tp2.16 Test for two-dimensional SMC propagation with Arctic pole handling.
ww3_tp2.17 Unstructured grid with Card Deck vs Domain Decomposition for Explicit vs Implicit schemes; grid integration (ww3_gint); grib2 outputs (ww3_grib).
ww3_tp2.18 Test for tidal constituents in ww3_prnc/ww3_prtide.
ww3_tp2.19 Unstructured grid with Domain Decomposition/Implicit schemes with Neumann Boundary Condition for Depth Breaking and Triad source terms (Boers 1996).
ww3_tp2.20 Vegetation source term (Anderson and Smith 2012).
ww3_tp2.21 Global Unstrucured Grid (periodicity and obstruction).
ww3_ts1 Source term test, time limited growth.
ww3_ts2 Source term test, fetch limited growth.
ww3_ts3 Source term test, hurricane with single moving grid.
ww3_ts4 Source term test, unresolved obstacles.
ww3_tic1.1 Wave-ice interaction, 1D test of S_{ice}.
ww3_tic1.2 Wave-ice interaction, 1D test of “shoaling” effect.
ww3_tic1.3 Wave-ice interaction, 1D test of refraction effect.
ww3_tic1.4 Wave-ice interaction, 1D test with ice floes and ice thickness.
ww3_tic2.1 Wave-ice interaction, 2D test of S_ice.
ww3_tic2.2 Wave-ice interaction, 2D test with non-uniform ice.
ww3_tic2.3 Wave-ice interaction, 2D test with uniform ice with increasing thickness.
ww3_tbt1.1 Wave-mud interaction, 1D test of S_mud.
ww3_tbt2.1 Wave-mud interaction, 2D test of S_mud.
ww3_tpt1.1 Tests for alternative spectral partitioning methods.
ww3_ta1 ww3_uprstrt, update the restart file of homogeneous conditions (1 point model).
mww3_test_01 Test for expanded grid mask with wetting and drying, etc.
mww3_test_02 Two-way nesting test with single inner grid.
mww3_test_03 Overlapping grids and two-way nesting tests (6-grid version with beach in high-resolution
mww3_test_04 Current or sea-mount test for two-way nesting with stationary swell conditions.
mww3_test_05 Three nested hurricane grids with moving grids test.
mww3_test_06 Tests for irregular grid(s) w/ ww3_multi.
mww3_test_07 Tests for unstructured grid(s) w/ ww3_multi.
mww3_test_08 Tests with wind and ice input.
ww3_ufs1.1 Tests for global 1deg grid w/ ww3_multi/ww3_multi_esmf; wind/current/ice inputs; ww3_grib outputs; grid integration (ww3_gint); restart/MPI task/thread reproducibility (ufs app).
ww3_ufs1.2 Tests for GFSv16 setup w/ ww3_multi/ww3_multi_esmf; wind/current/ice inputs; ww3_grib outputs; grid integration (ww3_gint); restart reproducibility (ufs app).

The above tests are contained in directories with the indicated tags within the regtest directory. A description of each test is summarized in the info file and required materials are in the input directories. A separate set of large input files should be downloaded from NOAA ftp site using the following script:

./model/bin/ww3_from_ftp.sh

Regression tests are run using the run_test script in the regtests/bin directory. How to run this script, including options, is shown by running run_test -h.

The Matrix: Comprehensive Regression Testing

A matrix of regression tests can be prepared in order to run a full suit of regression tests with different compiler options and HPC environment. This framework provides a way to run a larger set of regression tests, using a wider range of configurations. The practice is encouraged to ensure that changes made to the code will not unintentionally affect other parts of the code.

The list of required test can be modified in regtests/bin/matrix.base. Up to this point, there are two matrix_<machine> (matrix_ncep and matrix_datarmor) in regtests/bin/ to generate a matrix for the selected tests, defined in matrix.base. So far, these scripts are compatible with supercomputers used at NCEP and Ifremer/Datamor. Trusted institutions are encouraged to create their own versions of the matrix, which we will add to the bin directory in the future. The matrix_* scripts generate a batch-queue-ready script with the required modules for compilers, corresponding NetCDF and MPI libraries, and paths to METIS/PARMETIS/NC4, as well as the number of required cores used to execute the tests.

As part of the development framework in WAVEWATCH III®, prior to reintegrating feature/bugfix branches to the master or develop branches via a pull request, code managers at trusted institutions should check proposed code changes, comparing a matrix run made with the new branch with the stable develop branch, to ensure reproducibility of results and compilation. A set of scripts is available to assist in this process.

The following steps should be taken s part of the matrix comprehensive regression testing procedure:

  1. Clone the WW3 repository in (i.e. bf_branch or develop).
  2. Checkout the desired branch (develop/Master if the control or the feature or bugfix branch to be tested. Make sure the feature or bugfix branch is up to date with Develop.
  3. Download the large binary files from ftp site by executing ./model/bin/ww3_from_ftp.sh
  4. load required modules, required for w3_setup (i.e. compiler, MPI and NetCDF)
  5. set path to model/bin and model/exe directories.
  6. execute w3_setup <Home>/model to set the WW3 environment variables.
  7. Edit the matrix_<machine> to select the test list, required modules, set path and compiler option.
  8. generate matrix by executing matrix_<machine>.
  9. Submit the matrix.

The regression test matrix usually needs 14 hrs on NCEP’s R&D computer with 24 cores. If there is wall time limit (i.e.), the matrix is resubmitted. The submit script automatically skips tests that have been completed, and resumes the rest. This can be done in several steps if needed.

During runtime, a matrix log file is stored in a file determined by the wrapper script (default: matrix.out). This log file can be checked for compilation or other runtime errors by using a command such as “grep nri error matrix.out”. In case errors are found, which can be labeled “error”, “Error” and “ERROR” within matrix.out, the code manager/developer should debug the problem and repeat the process until a successful outcome is achieved.

After a successful matrix run, regression test outputs for the new feature/bugfix branch should be compared with the develop branch matrix outputs. The following steps are available:

  1. edit the path to the output of the regression test s for the develop branch in ./bin/matrix.comp
  2. execute ./bin/matrix.comp all, which compares all the regression tests and generates summary and full comparison.

If the differences are justified as a consequence of the new developments/changes in the feature branch, the code is ready for a pull request to the NOAA-EMC repo, or if from the latter, for a new release tag. If differences are unexpected, the proposed changes should be debugged until a successful outcome is attained.

Quick Links

Clone this wiki locally