Update gcycle to be able to read re-gridded GSI increment land files. #1010

ClaraDraper-NOAA · 2024-12-16T16:41:30Z

DESCRIPTION OF CHANGES:

Generalized the reading in of land increments to be:

soil increment OR snow increment file
on the GSI output grid to be regridded within global_cycle OR on the native model grid

Native model grid files may come from JEDI, or be from GSI and been regridded using a separate program. The read routine will detect which.

Also, changed the number of soil levels to be updated from 3 to 2. This will change the test output for all tests adding soil increments.

TESTS CONDUCTED:

If there are changes to the build or source code, the tests below must be conducted. Contact a repository manager if you need assistance.

[ hera only - I don't have access to the others] Compile branch on all Tier 1 machines using Intel (Orion, Jet, Hera, Hercules and WCOSS2).
Compile branch on Hera using GNU.
Compile branch in 'Debug' mode on WCOSS2.
Run unit tests locally on any Tier 1 machine.
Run global_cycle consistency tests locally on all Tier 1 machines.

Describe any additional tests performed.

DEPENDENCIES:

None.

DOCUMENTATION:

ISSUE:

addresses issue 1008

… grid.

ClaraDraper-NOAA · 2024-12-16T16:46:31Z

@GeorgeGayno-NOAA Submitting this as a draft for now, while I do the tests. I'm not sure which are the unit tests and which are the consistency tests?

I'm currently running rt.sh, with all the tests commented out except global_cycle.
I expect it to be zero-diff without the lsoil_incr change in global_cycle.sh, so am checking that, then will change the lsoil_incr.
I'll check it compiles with gnu on hera.
What other test should I do? ( note: I don't have access to any machines other than hera and gaeia)

GeorgeGayno-NOAA · 2024-12-16T18:01:07Z

@GeorgeGayno-NOAA Submitting this as a draft for now, while I do the tests. I'm not sure which are the unit tests and which are the consistency tests?

I'm currently running rt.sh, with all the tests commented out except global_cycle. I expect it to be zero-diff without the lsoil_incr change in global_cycle.sh, so am checking that, then will change the lsoil_incr. I'll check it compiles with gnu on hera. What other test should I do? ( note: I don't have access to any machines other than hera and gaeia)

The regression tests are in ./reg_tests/global_cycle. To run just the global_cycle test, you can invoke ./driver.hera.sh from the command line with no arguments. You don't need to bother with the ./rt.sh script.

The unit tests may be run by some slight modifications to the ./build_all.sh script:

 # The unit test data download is part of the build system. Not all machines can
 # access the EMC ftp site, so turn off the build (-DBUILD_TESTING=OFF) of the units tests accordingly.
 # Those with access to the EMC ftp site are: Orion and Hera.
-CMAKE_FLAGS+=" -DBUILD_TESTING=${BUILD_TESTING:-OFF}"
+CMAKE_FLAGS+=" -DBUILD_TESTING=${BUILD_TESTING:-ON}"

You may also need to update ./cmake/mpiexec.hera for an ACCOUNT you have access to.

ClaraDraper-NOAA · 2024-12-17T20:22:01Z

All 5 consistency checks passed on hera.
Compiled with gnu on hera.

ClaraDraper-NOAA · 2024-12-17T21:24:43Z

@GeorgeGayno-NOAA How do I confirm that the ctests passed. The first pass through, they didn't and I fixed it. build_all.sh compiled, but I can't find a report of the test results.

GeorgeGayno-NOAA · 2024-12-18T13:38:03Z

@GeorgeGayno-NOAA How do I confirm that the ctests passed. The first pass through, they didn't and I fixed it. build_all.sh compiled, but I can't find a report of the test results.

When I test your branch on Hera, I see one unit test failing:

      Start 26: global_cycle-ftst_read_increments
26/38 Test #26: global_cycle-ftst_read_increments ...........***Failed    3.74 sec

The test results will be in ./build/Testing/Temporary/LastTest.log. The test fails because of a missing file:

 FATAL ERROR: OPENING FILE: ./fnbgsi.001: No such file or directory
 STOP.

I can take a look.

ClaraDraper-NOAA · 2024-12-18T14:09:12Z

@GeorgeGayno-NOAA Is that from standard out? I think I'm not actually running the tests. Let me follow up on that.

GeorgeGayno-NOAA · 2024-12-18T14:21:01Z

@GeorgeGayno-NOAA Is that from standard out? I think I'm not actually running the tests. Let me follow up on that.

That is the output from the build_all.sh script after setting BUILD_TESTING=ON and uncommenting 'ctest' at the bottom.

ClaraDraper-NOAA · 2024-12-18T14:25:50Z

@GeorgeGayno-NOAA Is that from standard out? I think I'm not actually running the tests. Let me follow up on that.

That is the output from the build_all.sh script after setting BUILD_TESTING=ON and uncommenting 'ctest' at the bottom.

Just came here to say I found the ctest line! I'm running it now. I'll sort out the failing test.

ClaraDraper-NOAA · 2024-12-18T15:04:32Z

I fixed it. The reg_tests and ctests pass on hera, and gnu compilation works on hera. Converting from a draft. Yuan Xue and Tseganeh Gichamo would be good reviewers.

GeorgeGayno-NOAA

Do we need another regression test for a GSI case on the native grid?

i) write out the ensemble mean reanalysis, and apply it to the deterministic member. ii) replace UFS_UTILS regridding of Gaussian soil analysis to model grid with an external ESMF-based program (with appropriate land masking) Requires: sorc/gsi_enkf.fd develop after Tue Nov 12. Latest works. sorc/gdas.cd/sorc/da-utils PR: NOAA-EMC/DA-utils#5 sorc/ufs_utils.fd from PR: ufs-community/UFS_UTILS#1010

sorc/global_cycle.fd/cycle.f90

yuanxue2870 · 2024-12-23T15:32:50Z

Overall, looks good to me. Thanks!

GeorgeGayno-NOAA · 2025-01-13T16:16:10Z

@ClaraDraper-NOAA - make sure you have merged the latest updates from 'develop'.

tsga

Looks good to me.
@ClaraDraper-NOAA I am guessing the "gaussian_to_fv3_interp" function will soon be replaced by the ESMF re-gridding code you just wrote?

GeorgeGayno-NOAA · 2025-01-17T22:01:33Z

Do we need another regression test for a GSI case on the native grid?

To expand on my comment, there are two regression tests for applying land increments: C192.jedi_lndincsoilnoahmp.sh and C192.gsi_lndincsoilnoahmp.sh. The latter uses GSI data that needs to be re-gridded to the tiles. Should a test using tiled GSI data (no re-gridding necessary) be added? If you have a test case using tiled GSI data, I would be happy to convert it into a test.

GeorgeGayno-NOAA · 2025-01-17T22:15:19Z

Is logical do_lndinc really needed? Are do_soi_inc and do_sno_inc sufficient?

ClaraDraper-NOAA · 2025-01-21T20:29:24Z

@ClaraDraper-NOAA - make sure you have merged the latest updates from 'develop'.

done.

ClaraDraper-NOAA · 2025-01-21T20:30:59Z

Do we need another regression test for a GSI case on the native grid?

To expand on my comment, there are two regression tests for applying land increments: C192.jedi_lndincsoilnoahmp.sh and C192.gsi_lndincsoilnoahmp.sh. The latter uses GSI data that needs to be re-gridded to the tiles. Should a test using tiled GSI data (no re-gridding necessary) be added? If you have a test case using tiled GSI data, I would be happy to convert it into a test.

The read routine is slightly different for a re-gridded GSI file than for the two test cases that are already there. If you'd like me to prepare a new test, I can.

GeorgeGayno-NOAA · 2025-01-22T13:05:14Z

Do we need another regression test for a GSI case on the native grid?

To expand on my comment, there are two regression tests for applying land increments: C192.jedi_lndincsoilnoahmp.sh and C192.gsi_lndincsoilnoahmp.sh. The latter uses GSI data that needs to be re-gridded to the tiles. Should a test using tiled GSI data (no re-gridding necessary) be added? If you have a test case using tiled GSI data, I would be happy to convert it into a test.

The read routine is slightly different for a re-gridded GSI file than for the two test cases that are already there. If you'd like me to prepare a new test, I can.

Yes. Please do.

…l states.

ClaraDraper-NOAA · 2025-01-24T17:55:10Z

Looks good to me. @ClaraDraper-NOAA I am guessing the "gaussian_to_fv3_interp" function will soon be replaced by the ESMF re-gridding code you just wrote?

This PR effectively adds the option to do either. However, the ESMF regridding is more appropriate - so perhaps in the future we can just delete the gaussian_to_fv3_interp as you suggest.

ClaraDraper-NOAA · 2025-01-24T18:01:29Z

Do we need another regression test for a GSI case on the native grid?

To expand on my comment, there are two regression tests for applying land increments: C192.jedi_lndincsoilnoahmp.sh and C192.gsi_lndincsoilnoahmp.sh. The latter uses GSI data that needs to be re-gridded to the tiles. Should a test using tiled GSI data (no re-gridding necessary) be added? If you have a test case using tiled GSI data, I would be happy to convert it into a test.

The read routine is slightly different for a re-gridded GSI file than for the two test cases that are already there. If you'd like me to prepare a new test, I can.

Yes. Please do.

I just added a new test, and have checked it passes on hera. I added the new test to the drivers for the other machines, except s4 which looks a bit different to the others.

The additional files that you need to add to the COMIN are:
/scratch2/BMC/gsienkf/Clara.Draper/gerrit-hera/PRprep/NEWTEST_UFS_UTILS/UFS_UTILS_COMIN/soil_xainc*

Examples of the baseline files are here:
/scratch2/BMC/gsienkf/Clara.Draper/gerrit-hera/PRprep/NEWTEST_UFS_UTILS/NEW_BASELINE_FILES/c192.gsitile_lndincsoilnoahmp

ClaraDraper-NOAA · 2025-01-24T18:06:59Z

Is logical do_lndinc really needed? Are do_soi_inc and do_sno_inc sufficient?

@GeorgeGayno-NOAA Yes, do_lndinc is redundant, since it can be turned on by the do_*_inc options. However, my preference would be to keep it (together with the DO_SFCCYCLE and DONST) to make it clear which actions (adding land increments, updating land model parameters, and/or updating NST) cycle if being called for. It's up to you though - if you'd like me to remove it, let me know. We could either completely remove it, or remove it from the namelists, and set it internally.

ClaraDraper-NOAA · 2025-01-24T18:07:55Z

@GeorgeGayno-NOAA I think I've addressed everything, let me know if there's anything else I can do.

GeorgeGayno-NOAA · 2025-01-27T13:47:39Z

Is logical do_lndinc really needed? Are do_soi_inc and do_sno_inc sufficient?

@GeorgeGayno-NOAA Yes, do_lndinc is redundant, since it can be turned on by the do_*_inc options. However, my preference would be to keep it (together with the DO_SFCCYCLE and DONST) to make it clear which actions (adding land increments, updating land model parameters, and/or updating NST) cycle if being called for. It's up to you though - if you'd like me to remove it, let me know. We could either completely remove it, or remove it from the namelists, and set it internally.

Ok. We can keep it. But I would update the script and code prologs, so the general user knows how to set each variable. For example, I am guessing if 'do_soi_inc' is set to true, then 'do_lndinc' must also be set to true.

GeorgeGayno-NOAA · 2025-01-27T13:48:28Z

ush/global_cycle.sh

-#     DO_SOI_INC_GSI    Call routine to update soil states with gsi(gaussian) increment files
-#     DO_SNO_INC_JEDI   Call routine to update snow states with jedi increment files
-#     DO_SOI_INC_JEDI   Call routine to update soil states with jedi increment files
+#     DO_SOI_INC    Call routine to update soil states 


I would add more detail on how to set each logical.

I've updated the comments, and added some logic into the global_driver.sh to set DO_LNDINC to true if either of DO_SNO_INC or DO_SOI_INC it true.

GeorgeGayno-NOAA · 2025-01-27T13:49:29Z

Do we need another regression test for a GSI case on the native grid?

To expand on my comment, there are two regression tests for applying land increments: C192.jedi_lndincsoilnoahmp.sh and C192.gsi_lndincsoilnoahmp.sh. The latter uses GSI data that needs to be re-gridded to the tiles. Should a test using tiled GSI data (no re-gridding necessary) be added? If you have a test case using tiled GSI data, I would be happy to convert it into a test.

The read routine is slightly different for a re-gridded GSI file than for the two test cases that are already there. If you'd like me to prepare a new test, I can.

Yes. Please do.

I just added a new test, and have checked it passes on hera. I added the new test to the drivers for the other machines, except s4 which looks a bit different to the others.

The additional files that you need to add to the COMIN are: /scratch2/BMC/gsienkf/Clara.Draper/gerrit-hera/PRprep/NEWTEST_UFS_UTILS/UFS_UTILS_COMIN/soil_xainc*

Examples of the baseline files are here: /scratch2/BMC/gsienkf/Clara.Draper/gerrit-hera/PRprep/NEWTEST_UFS_UTILS/NEW_BASELINE_FILES/c192.gsitile_lndincsoilnoahmp

Great. I will host these files in the official location on each machine.

GeorgeGayno-NOAA · 2025-01-27T13:51:03Z

Do we need another regression test for a GSI case on the native grid?

To expand on my comment, there are two regression tests for applying land increments: C192.jedi_lndincsoilnoahmp.sh and C192.gsi_lndincsoilnoahmp.sh. The latter uses GSI data that needs to be re-gridded to the tiles. Should a test using tiled GSI data (no re-gridding necessary) be added? If you have a test case using tiled GSI data, I would be happy to convert it into a test.

The read routine is slightly different for a re-gridded GSI file than for the two test cases that are already there. If you'd like me to prepare a new test, I can.

Yes. Please do.

I just added a new test, and have checked it passes on hera. I added the new test to the drivers for the other machines, except s4 which looks a bit different to the others.

Don't worry about s4. That is not an officially supported machine. Only WCOSS2, Jet, Hera, Hercules and Orion are supported.

The additional files that you need to add to the COMIN are: /scratch2/BMC/gsienkf/Clara.Draper/gerrit-hera/PRprep/NEWTEST_UFS_UTILS/UFS_UTILS_COMIN/soil_xainc*

Examples of the baseline files are here: /scratch2/BMC/gsienkf/Clara.Draper/gerrit-hera/PRprep/NEWTEST_UFS_UTILS/NEW_BASELINE_FILES/c192.gsitile_lndincsoilnoahmp

GeorgeGayno-NOAA · 2025-01-27T16:15:33Z

I placed the new baseline files on Hera:

[role.ufsutils@hfe11 c192.gsitile_lndincsoilnoahmp]$ pwd
/scratch1/NCEPDEV/nems/role.ufsutils/ufs_utils/reg_tests/global_cycle/baseline_data/c192.gsitile_lndincsoilnoahmp
[role.ufsutils@hfe11 c192.gsitile_lndincsoilnoahmp]$ ls -l
total 223104
-r--r--r-- 1 role.ufsutils nems 38073028 Jan 27 16:08 20190730.000000.sfcanl_data.tile1.nc
-r--r--r-- 1 role.ufsutils nems 38073028 Jan 27 16:08 20190730.000000.sfcanl_data.tile2.nc
-r--r--r-- 1 role.ufsutils nems 38073028 Jan 27 16:08 20190730.000000.sfcanl_data.tile3.nc
-r--r--r-- 1 role.ufsutils nems 38073028 Jan 27 16:08 20190730.000000.sfcanl_data.tile4.nc
-r--r--r-- 1 role.ufsutils nems 38073028 Jan 27 16:08 20190730.000000.sfcanl_data.tile5.nc
-r--r--r-- 1 role.ufsutils nems 38073028 Jan 27 16:08 20190730.000000.sfcanl_data.tile6.nc

and the new input files here:

/scratch1/NCEPDEV/nems/role.ufsutils/ufs_utils/reg_tests/global_cycle/input_data_noahmp
[role.ufsutils@hfe11 input_data_noahmp]$ ls -l soil_xainc.*
-rw-r--r-- 1 role.ufsutils nems 1187094 Jan 27 15:59 soil_xainc.gsi.tile1.nc.pr1010
-rw-r--r-- 1 role.ufsutils nems 1187094 Jan 27 15:59 soil_xainc.gsi.tile2.nc.pr1010
-rw-r--r-- 1 role.ufsutils nems 1187094 Jan 27 15:59 soil_xainc.gsi.tile3.nc.pr1010
-rw-r--r-- 1 role.ufsutils nems 1187094 Jan 27 15:59 soil_xainc.gsi.tile4.nc.pr1010
-rw-r--r-- 1 role.ufsutils nems 1187094 Jan 27 15:59 soil_xainc.gsi.tile5.nc.pr1010
-rw-r--r-- 1 role.ufsutils nems 1187094 Jan 27 15:59 soil_xainc.gsi.tile6.nc.pr1010
lrwxrwxrwx 1 role.ufsutils nems      30 Jan 27 16:03 soil_xainc.tile1.nc -> soil_xainc.gsi.tile1.nc.pr1010
lrwxrwxrwx 1 role.ufsutils nems      30 Jan 27 16:03 soil_xainc.tile2.nc -> soil_xainc.gsi.tile2.nc.pr1010
lrwxrwxrwx 1 role.ufsutils nems      30 Jan 27 16:03 soil_xainc.tile3.nc -> soil_xainc.gsi.tile3.nc.pr1010
lrwxrwxrwx 1 role.ufsutils nems      30 Jan 27 16:04 soil_xainc.tile4.nc -> soil_xainc.gsi.tile4.nc.pr1010
lrwxrwxrwx 1 role.ufsutils nems      30 Jan 27 16:04 soil_xainc.tile5.nc -> soil_xainc.gsi.tile5.nc.pr1010
lrwxrwxrwx 1 role.ufsutils nems      30 Jan 27 16:04 soil_xainc.tile6.nc -> soil_xainc.gsi.tile6.nc.pr1010

I will push the data to the other machines after we verify everything is working.

ClaraDraper-NOAA added 3 commits December 15, 2024 22:37

Updates to generalize inputs land increments to Gaussian grid and fv3…

75c736e

… grid.

Merge branch 'develop' into feature/gcycle_fv3incr

1a140ab

Udpated namelist variables in gcycle tests.

8fedf3d

GeorgeGayno-NOAA self-requested a review December 16, 2024 17:48

ClaraDraper-NOAA added 2 commits December 17, 2024 19:22

update reg_test namelist to use new variables.

a635ce7

Added LSOIL_INCR variable.

633f9bf

Update to test for new read_data arguments.

ddde3f3

Fixed filename in ctest for soil increment.

2a4848d

ClaraDraper-NOAA marked this pull request as ready for review December 18, 2024 15:05

GeorgeGayno-NOAA requested a review from yuanxue2870 December 18, 2024 15:34

GeorgeGayno-NOAA reviewed Dec 18, 2024

View reviewed changes

yuanxue2870 reviewed Dec 19, 2024

View reviewed changes

sorc/global_cycle.fd/cycle.f90 Outdated Show resolved Hide resolved

GeorgeGayno-NOAA requested a review from tsga January 13, 2025 16:16

tsga approved these changes Jan 14, 2025

View reviewed changes

Merge branch 'develop' into feature/gcycle_fv3incr

c66ed03

Added new test for adding pre-interpolated increments from GSI to soi…

0d14319

…l states.

Tidy up comments.

9f78469

GeorgeGayno-NOAA reviewed Jan 27, 2025

View reviewed changes

Clarified do_lndinc, do_soi_inc, and do_sno_inc in comments and script.

76bc5a9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update gcycle to be able to read re-gridded GSI increment land files. #1010

Update gcycle to be able to read re-gridded GSI increment land files. #1010

ClaraDraper-NOAA commented Dec 16, 2024 •

edited by GeorgeGayno-NOAA

Loading

ClaraDraper-NOAA commented Dec 16, 2024

GeorgeGayno-NOAA commented Dec 16, 2024

ClaraDraper-NOAA commented Dec 17, 2024

ClaraDraper-NOAA commented Dec 17, 2024

GeorgeGayno-NOAA commented Dec 18, 2024

ClaraDraper-NOAA commented Dec 18, 2024

GeorgeGayno-NOAA commented Dec 18, 2024

ClaraDraper-NOAA commented Dec 18, 2024

ClaraDraper-NOAA commented Dec 18, 2024

GeorgeGayno-NOAA left a comment

yuanxue2870 commented Dec 23, 2024

GeorgeGayno-NOAA commented Jan 13, 2025

tsga left a comment •

edited

Loading

GeorgeGayno-NOAA commented Jan 17, 2025

GeorgeGayno-NOAA commented Jan 17, 2025

ClaraDraper-NOAA commented Jan 21, 2025

ClaraDraper-NOAA commented Jan 21, 2025

GeorgeGayno-NOAA commented Jan 22, 2025

ClaraDraper-NOAA commented Jan 24, 2025

ClaraDraper-NOAA commented Jan 24, 2025

ClaraDraper-NOAA commented Jan 24, 2025

ClaraDraper-NOAA commented Jan 24, 2025

GeorgeGayno-NOAA commented Jan 27, 2025

GeorgeGayno-NOAA Jan 27, 2025

ClaraDraper-NOAA Jan 27, 2025

GeorgeGayno-NOAA commented Jan 27, 2025

GeorgeGayno-NOAA commented Jan 27, 2025

GeorgeGayno-NOAA commented Jan 27, 2025

Update gcycle to be able to read re-gridded GSI increment land files. #1010

Are you sure you want to change the base?

Update gcycle to be able to read re-gridded GSI increment land files. #1010

Conversation

ClaraDraper-NOAA commented Dec 16, 2024 • edited by GeorgeGayno-NOAA Loading

DESCRIPTION OF CHANGES:

TESTS CONDUCTED:

DEPENDENCIES:

DOCUMENTATION:

ISSUE:

ClaraDraper-NOAA commented Dec 16, 2024

GeorgeGayno-NOAA commented Dec 16, 2024

ClaraDraper-NOAA commented Dec 17, 2024

ClaraDraper-NOAA commented Dec 17, 2024

GeorgeGayno-NOAA commented Dec 18, 2024

ClaraDraper-NOAA commented Dec 18, 2024

GeorgeGayno-NOAA commented Dec 18, 2024

ClaraDraper-NOAA commented Dec 18, 2024

ClaraDraper-NOAA commented Dec 18, 2024

GeorgeGayno-NOAA left a comment

Choose a reason for hiding this comment

yuanxue2870 commented Dec 23, 2024

GeorgeGayno-NOAA commented Jan 13, 2025

tsga left a comment • edited Loading

Choose a reason for hiding this comment

GeorgeGayno-NOAA commented Jan 17, 2025

GeorgeGayno-NOAA commented Jan 17, 2025

ClaraDraper-NOAA commented Jan 21, 2025

ClaraDraper-NOAA commented Jan 21, 2025

GeorgeGayno-NOAA commented Jan 22, 2025

ClaraDraper-NOAA commented Jan 24, 2025

ClaraDraper-NOAA commented Jan 24, 2025

ClaraDraper-NOAA commented Jan 24, 2025

ClaraDraper-NOAA commented Jan 24, 2025

GeorgeGayno-NOAA commented Jan 27, 2025

GeorgeGayno-NOAA Jan 27, 2025

Choose a reason for hiding this comment

ClaraDraper-NOAA Jan 27, 2025

Choose a reason for hiding this comment

GeorgeGayno-NOAA commented Jan 27, 2025

GeorgeGayno-NOAA commented Jan 27, 2025

GeorgeGayno-NOAA commented Jan 27, 2025

ClaraDraper-NOAA commented Dec 16, 2024 •

edited by GeorgeGayno-NOAA

Loading

tsga left a comment •

edited

Loading