-
Notifications
You must be signed in to change notification settings - Fork 108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add capability to use RAP/HRRR MASSDEN as smoke IC/LBCs #923
base: develop
Are you sure you want to change the base?
Conversation
I am curious what project or implementation this PR supports. |
I'm not sure exactly which details you need, but this is largely research oriented e.g., For computational efficiency, for retrospective experiments prior to RRFS-A SD implementation, we often only want to run the CONUS domain but would still like e.g., Canadian originated smoke to enter the domain (e.g., DRSA FIRE3). The same could be said for using HRRR IC/BCs for the fire nest. This wIll also support NOAA ERB funded work. |
We ask that developers run the consistency tests located in |
Will do - and will look into these failures. Thank you for the information. |
I ran the tests on Jet and also received the two failures (13-km CONUS RAP W/ GSD PHYSICS AND SFC FROM FILE GRIB2 and 3-km CONUS HRRR W/ GSD PHYSICS AND SFC FROM FILE GRIB2). This makes sense though, since smoke is being added to the files and so e.g., there is an additional tracer in the out files. How would you like this to be addressed? |
The number of tracers in |
As a test, I ran the The test script was updated as follows:
The modified input data and the script used to create it is here: |
But if the end result of my modifications is to add a new tracer to the out file, won't it automatically fail? |
For the first test using the modified input data, I updated the varmap file as follows:
Since there are no smoke records, processing should stop. And that is what the log file shows:
|
I don't have access to this system. Are you suggesting I add something to the code? Or the reg test? |
Can you explain why you modified this? |
The varmap files have several choices with how to process fields:
According to the README file, users can adjust how fields are processed using the varmap file. (See the instructions for column 3). Options include 'stop', 'skip' 'set_to_fill' etc. These different options need to be tested. So, I am temporarily modifying the varmap file. https://github.com/ufs-community/UFS_UTILS/blob/develop/parm/varmap_tables/README |
Next, test the 'skip' function. Modify the varmap table as follows:
The log file shows:
And the out.atm.tile7.nc file contains a smoke record. Because the input GRIB2 file does not contain smoke, that was unexpected. According to the README, when 'skip' is used, there should not be a smoke record.
@LarissaReames-NOAA and @JeffBeck-NOAA - you understand the GRIB2 option better I do. What is the expected result in this case? |
@jordanschnell - this PR has not been updated in a while. There is more work required if you want to merge it. What do you want to do? |
@GeorgeGayno-NOAA - We would like to merge it, however, I was not sure what else is required. Your question to Jeff and Larissa went unanswered and I am unsure of what the issue is. It seems like there may be other issues though (#851) |
I think what @jordanschnell needs to do is create his own var mapping file based on the GSD var mapping file he's tested with. I don't think we should be modifying a file we've long used in the current form, especially when this doesn't reflect some operational change. This way there won't be any failures in the regression tests either. How you'd choose the option for missing variables really depends. We never implemented any behavior to not write an array to file if "skip" is chosen and the variable is unavailable in the grib2 file. I believe the philosophy was always that, given a certain weather model SDF, we always wanted the atm init files to have certain arrays in them regardless of what data is available in the grib2 files used to create them. If I remember correctly, we use "skip" for those variables that might be missing data at certain levels, like RH in some GFS grib2 files, so that a few missing levels wouldn't cause a failure. If you plan on a variable possible being completely missing from a grib2 file but don't want that to cause a failure, I'd suggest using set_to_fill. So, in that sort of paradigm, one might expect any weather model run utilizing HRRR Smoke to also require its own unique SDF, so it would make sense to have a separate variable mapping file. In that mapping file you could indeed set the missing behavior to be "stop" to make sure a user is initializing that weather model SDF with all the desired smoke tracers. |
…from the original varmap
@LarissaReames-NOAA @GeorgeGayno-NOAA I have created a separate varmap file that contains smoke and removed smoke from the original file. To use, the varmap file in make_ics/make_lbcs scripts needs to be set to the new file. Please let me know if you would like further modifications. |
@jordanschnell - be sure to merge updates from develop to your branch. |
@GeorgeGayno-NOAA, is that just:
Then do I need to |
That will work if 'develop' in your fork is up-to-date with the authoritative fork. See our wiki page for how I do it (last two bullets). https://github.com/ufs-community/UFS_UTILS/wiki/3.-Checking-out-code-and-making-changes I like to do things from the command line. But, I would guess you can also do these updates from the web browser. |
Smoke is in the output (/lfs5/BMC/rtwbl/rap-chem/tmp/regtest_out/reg-tests/chgres-cube/13km_conus_rap-smoke_grib2), but the regtest failed because it is not in the baseline. |
I created a baseline directory. Update the
|
I created baseline directories on all Tier 1 machines. |
Great. Do you have access to the other Tier 1 machines? Hera, Orion, Hercules and WCOSS2? If not, I can update the driver scripts on those machines and run the tests. |
Because you are adding a varmap file, this section of the documentation may need some updates: I can help with that. |
@GeorgeGayno-NOAA I have access to Hera, but not the others. so, these are my remaining tasks?
|
And do as many of the tests described above in "TESTS CONDUCTED". I can do the tests on the Herc, Orion and WCOSS2. |
Regtest on Hera passed: /scratch1/BMC/wrf-chem/Jordan/RRFS-SD/src/PRs/rap_icbc/rrfs-workflow/sorc/UFS_UTILS/reg_tests/chgres_cube/consistency.log14 |
@jordanschnell - I updated and tested the regression test driver scripts on Hercules, Orion and WCOSS2. I placed them on Hera here: /scratch1/NCEPDEV/da/George.Gayno/ufs_utils.git/UFS_UTILS.schnell/reg_tests/chgres_cube They have ".updated" in their filename. |
Great thank you! So commit these? |
Yes. |
@jordanschnell - Once you update the documentation, I will approve and merge this PR. |
Great - how exactly do I update this? As for specifically what to add, are you thinking I add a note under the Column 1, something to the effect of: *** If smoke from RAP/HRRR is desired, one must select the GSDphys_smoke_var_map.txt varmap file which includes massden - smoke mass mixing ratio |
I set up readthedocs for your branch here: You will edit the ./docs/source/chgres_cube.rst file. When you push your branch to your fork, the new documentation should automatically update. Yes, I would add the definition of massden to the list. Try that first and see if everything works. Then, we can add additional instructions for users. |
@GeorgeGayno-NOAA - Commit is pushed, but I don't see it in the docs branch you created |
Looks like I have to run it manually. Take a look now. The formatting does not look right. |
I would just add the definition of massden. |
ok hopefully that worked |
The format looks good. One more favor, can you remove the line: "(Note, only the GFS physics suite is currently supported.)" and add "Each file contains the following columns:" |
Should be good to go! |
DESCRIPTION OF CHANGES:
Previously, massden/smoke was not part of the variable list to ingest into IC/LBCs from RAP/HRRR models. This PR adds the massden/smoke to the mapping table and adds the necessary functionality to read in and convert to the correct units. An additional allocated array is created when using RAP/HRRR for ICs/LBCs during the smoke read to be able to convert units. Other models (e.g., GFS) will not attempt to read in smoke.
TESTS CONDUCTED:
chgres_cube
consistency tests locally on all Tier 1 machines. Done using 0a954d1 on Jet, Orion, Hercules and Hera.Additional tests performed.
DEPENDENCIES:
none
DOCUMENTATION:
Update readthedocs: https://schnel.readthedocs.io/en/rap_icbc/ufs_utils.html#variable-mapping-varmap-table
ISSUE:
Fixes #851.
CONTRIBUTORS:
Johana Romero-Alvarez (CIRES/NOAA GSL)