Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Land DA container to Spack-Stack 1.6.0 #147

Merged

Conversation

EdwardSnyder-NOAA
Copy link
Collaborator

@EdwardSnyder-NOAA EdwardSnyder-NOAA commented Oct 4, 2024

Description

This PR updates or adds files to support updating the Land DA container to spack-stack 1.6.0. Major changes include adding modulefile task files for singularity, added a singularity land_analysis file, and document changes.

The Land DA container was built with this PR using this landda dockerfile which inherits this spack-stack dockerfile. Testing was done with this container using the land_analysis test described in the docs which passed on GCP, AWS, and Hercules.

UPDATE:
The latest Land DA container was built with the updated wm hash, which required updates to the landda dockerfile and spack-stack dockerfile. Here is the setup_container.sh script that is used in this process. Lastly, the ctest Dockerfile was tested with our Jenkins pipeline and passes without any problems.

Anticipated changes to regression tests:

  • Is baseline change expected ?

Subcomponents involved:

  • apply_incr.fd (NOAA-PSL/land-apply_jedi_incr)
  • ufs_model.fd (ufs-community/ufs-weather-model)
  • none

Linked PR's and Issues:

Testing (for CM's):

  • RDHPCS
    • Hera
    • Orion
    • Hercules
  • CI
    • Completed
  • WE2E
    • Completed
  • PW-Clouds
    • AWS
    • AZURE
    • GCP

@EdwardSnyder-NOAA EdwardSnyder-NOAA marked this pull request as ready for review October 8, 2024 16:54
@jkbk2004
Copy link
Collaborator

@EdwardSnyder-NOAA
Copy link
Collaborator Author

Yes, @jkbk2004 I'll need to update that file. I got it to work but had to make a number of changes that I think could be resolved in a more elegant way. I'll try to get those changes in today.

Just a reminder this PR is for the previous ufs model hash. I'm still working on building the spack-stack with the proper modules to support the new ufs model hash.

@EdwardSnyder-NOAA
Copy link
Collaborator Author

The container has been updated to work with the new ufs wm hash. All the scripts are in order and it runs successfully on AWS (/contrib/Edward.Snyder/landda-container/aug-update/wm-hash-update/updated/land-DA_workflow):

[Edward.Snyder@aws-202 parm]$ rocotostat -w land_analysis.xml -d land_analysis.db
       CYCLE                    TASK                       JOBID               STATE         EXIT STATUS     TRIES      DURATION
================================================================================================================================
200001030000                prep_obs                          54           SUCCEEDED                   0         1           9.0
200001030000                pre_anal                          55           SUCCEEDED                   0         1          17.0
200001030000                analysis                          57           SUCCEEDED                   0         1          26.0
200001030000               post_anal                          58           SUCCEEDED                   0         1           9.0
200001030000                forecast                          59           SUCCEEDED                   0         1          40.0
200001030000              plot_stats                          60           SUCCEEDED                   0         1          59.0
================================================================================================================================
200001040000                prep_obs                          56           SUCCEEDED                   0         1           9.0
200001040000                pre_anal                          61           SUCCEEDED                   0         1           8.0
200001040000                analysis                          62           SUCCEEDED                   0         1          19.0
200001040000               post_anal                          63           SUCCEEDED                   0         1           8.0
200001040000                forecast                          64           SUCCEEDED                   0         1          38.0
200001040000              plot_stats                          65           SUCCEEDED                   0         1          25.0

The ctest Dockerfile also works, which was tested with our Jenkins pipeline:

#13 57.04 100% tests passed, 0 tests failed out of 4
#13 57.04 
#13 57.04 Total Test time (real) =  55.52 sec
#13 DONE 70.2s`

@chan-hoo chan-hoo merged commit 597c38d into ufs-community:develop Oct 25, 2024
2 checks passed
@chan-hoo chan-hoo linked an issue Oct 25, 2024 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Need to update container capability
3 participants