ufs-bundle update to support building ATM/NG-GODAS/S2S UFS, and running ATM warmstart configurations

[climbfuji]

Description

This PR includes all the contributions from @mark-a-potts described in PRs #3 and #5, and on top of it many more updates that enable building the ufs-bundle in atmosphere-only (ATM), ocean-only (NG-GODAS) and fully coupled (S2S) configurations.

Due to recent updates to the ufs-weather-model (and in particular mom6, cice6), the NG-GODAS ctest is no longer working (need update to input data sets, amongst others). There is no ctest for the S3S configuration yet. For atmosphere-only, we have two working ctests for setting up warmstart and coldstart run directories, and two working warmstart runs (model and forecast). The coldstart runs need more work to enable reading UFS coldstart initial conditions in fv3-jedi.

Definition of Done

Can we merge this PR and mark it as completed, even though the code changes for the submodules are still in their own branches and not yet merged?

Issue(s) addressed

Fixes #12
Working towards #8
Working towards #15
Working towards #16

Dependencies

JEDI

https://github.com/JCSDA-internal/fv3-jedi-linearmodel/pull/16
https://github.com/JCSDA-internal/fv3-jedi/pull/697
JCSDA-internal/soca#824
JCSDA-internal/fv3-jedi-data#52
https://github.com/JCSDA-internal/saber/pull/385

UFS

NOAA-EMC/AQM#40
NOAA-EMC/MOM6#107
NOAA-GFDL/GFDL_atmos_cubed_sphere#231
NOAA-EMC/fv3atm#620
ufs-community/ufs-weather-model#1504

Impact

None

[shlyaeva]

One quick comment on looking through the code changes, and I'll also build and test today.

[shlyaeva]

I've started on orion which doesn't have fms/2022.04 yet. Is there an easy way to build it as part of the bundle? No worries if not, I don't have to build it there. If not, do you have a recommended platform to test (pcluster?)

[climbfuji]

(1) You should be able to build a new spack-stack environment from the latest spack-stack develop branch using the skylab-dev template
(2) You should also be able to add fms@2022.04 to your existing skylab-v3 environment (but take a backup first, just in case), even without updating the spack-stack and spack code that you checked out for v3. After activating your existing environment:

spack add fms@2022.04 +64bit +quad_precision +gfs_phys +openmp +fpic constants=GFS
spack concretize
spack install --verbose --source
spack module lmod refresh fms

After loading the compiler, mpi and python metamodules, load jedi-ufs-env followed by loading fms@2022.04 (which will swap out the already loaded fms@2022.02

[shlyaeva]

I built on my mac (with freshly built spack-stack from yesterday's develop using skylab-dev template) and ran fv3-jedi tests (only for ATM). I am approving based on this (I don't have expertise to review cmake changes). Some fv3-jedi tests are failing, but I think it's a known issue (lam tests). I'm starting to update JEDI components to develop.

[shlyaeva]

This readme is super informative and useful, made it so easy to build. Thank you in particular for notes on building on mac!

[shlyaeva]

Some fv3-jedi tests are failing, but I think it's a known issue (lam tests)

Actually, I'm not sure it's a known issue. @climbfuji do lam_cmaq tests pass for you with this build?

[climbfuji]

Some fv3-jedi tests are failing, but I think it's a known issue (lam tests)

Actually, I'm not sure it's a known issue. @climbfuji do lam_cmaq tests pass for you with this build?

No, they don't. That's something we need to figure out. But because this is the ufs-bundle and not the fv3-bundle, I am unclear which tests have to pass (and which can fail for what reason).

[climbfuji]

This is your weekly, friendly reminder ... can we get this reviewed please? Thanks!

[shlyaeva]

@mark-a-potts, @aerorahul, @cmgas do you have time to review in the next couple of days?

[aerorahul]

The changes to the CMakeLists.txt look reasonable.
The addition of README.md is much appreciated.

[shlyaeva]

Thank you so much @aerorahul!

[cmgas]

I used this branch to build on my Mac and all is going well, thank you! And great ReadMe :)

[climbfuji]

Thanks everyone, I think we should merge this to get it out of the way and then work on the outstanding issues?

[shlyaeva]

I agree with @climbfuji