POD discussion: AMOC 3D structure diagnostic #118

tsjackson-noaa · 2020-02-18T18:59:29Z

tsjackson-noaa
Feb 18, 2020

Thread for progress in adding AMOC 3D structure diagnostic from Xiaobiao Xu (FSU/COAPS).

Legacy documentation:
http://www.cgd.ucar.edu/cms/bundy/Projects/diagnostics/mdtf/mdtf_figures/MDTF_GFDL-CM2p1/transport_onto_TS/transport_onto_TS.html

tsjackson-noaa · 2020-08-14T19:56:55Z

tsjackson-noaa
Aug 14, 2020
Author

Progress update:

Link to development branch and POD commit history. These links will stop working when the branch is deleted after integration work is finished. Links to files below will remain valid.

Current bugs/errors:

None at present; POD is at pre-test stage.

Remaining necessary tasks:

Framework file search logic needs to be updated to handle data at fx/static frequency, specifically the regexes for the GFDL postprocessing directory hierarchy and the filename regex for the CMIP6 DRS.

0 replies

tsjackson-noaa · 2020-09-13T20:40:21Z

tsjackson-noaa
Sep 13, 2020
Author

The following warnings raised in the logs are believed to be a result of normal operation and won't be fixed:

CDO: The CDO utilities are called by

MDTF-diagnostics/diagnostics/AMOC_3D_Structure/interp_vit_to_viv_monthly.ncl

Lines 55 to 61 in 8ecb0ac

    
           fo=DIR_out+model+"."+thetao_var+"_viv_"+yymm+".mon.nc" 
        
           system("rm -f "+fo) 
        
           system("cdo --no_warnings -ifthen "+ncI+" -remapbil,"+ncI+" -fillmiss2 -setctomiss,0 -selvar,"+thetao_var+" -seltimestep,"+step0+"/"+step1+" "+ncT+" "+fo) 
        
           fo=DIR_out+model+"."+so_var+"_viv_"+yymm+".mon.nc" 
        
           system("rm -f "+fo) 
        
           system("cdo --no_warnings -ifthen "+ncI+" -remapbil,"+ncI+" -fillmiss2 -setctomiss,0 -selvar,"+so_var+" -seltimestep,"+step0+"/"+step1+" "+ncS+" "+fo)

which raises Warning (cdfScanVarAttr): NetCDF: Variable not found - lat_vertices and other warnings for nonstandard coordinate data included in the sample model data. This warning message seems to be emitted whenever CDO finds coordinate data outside of the time/level/lat/lon model, see eg https://code.mpimet.mpg.de/boards/1/topics/1168. We assume that lat_vertices and the other variables aren't being used downstream.

NCL: Calls to dim_avg_n_Wrap in Qavg_lats_y_plot.ncl, Swgt_lats_y_plot.ncl and Twgt_lats_y_plot.ncl raise warning:Dimension (0) has not been defined. Believe this doesn't affect data processing and that the warning is issued because no name is associated with dimension 0 (https://mailman.ucar.edu/pipermail/ncl-talk/2017-February/008053.html).

0 replies

tsjackson-noaa · 2020-09-13T20:55:33Z

tsjackson-noaa
Sep 13, 2020
Author

The POD's code requires no changes to fit in with the digested observational data policy: included obs data is only 138 MB, and obs data is only used to generate plots (obs_AMOC_3D_Structure.py and the NCL routines called from there.)

One of the changes made to the framework since this POD was submitted is that analysis of different models is handled by the framework, not the POD (in other words, each POD processes only one experimental run when it's called.) This means that the following plots linked to from AMOC_3D_Structure.html are redundant, since they contain observational data only:

"Observations or High-res model" row: streamfunction, θ(z), S(z), Ψ_σ, Ψ_θ, Ψ_S
Latitudinal variation "High-res Model" row: Layer Q(y), θ(y), S(y); Heat Transport, Freshwater transport.

The plots will continue to be generated to minimize changes to the existing code.

0 replies

tsjackson-noaa · 2020-09-14T17:37:55Z

tsjackson-noaa
Sep 14, 2020
Author

The following issues remain and can't be addressed without input from the POD authors:

1. CMIP5 climatology data. Climatology data was included for 20 models in the CMIP5 ensemble, but some summary plots make reference to data that wasn't provided. It's unclear whether this data was intended to be computed by the POD, or is missing. Included files are *.AMOC_qts.clim.nc, *.MFWT.clim.nc, *.MHT.clim.nc, *.so_wgt.clim.nc, *.thetao_wgt.clim.nc; files which may be missing are *.AMOC_T2B.clim.nc, *.trans_avg.clim.nc, *.lat_vo.fx.nc.

2. Malformed scatter plot. The plot produced by Twgt_vs_Swgt_lat0_dots_plot.ncl appears incorrect: data is plotted twice on the same set of axes. The source code has many lines commented out which indicate that it may have been intended to be a multi-panel plot, but it's unclear what was intended.

3. Nonexistent rows in climatology data. sum_MFWT_lats_y_plot.ncl and sum_MHT_lats_y_plot.ncl reference a row in the CMIP5 climatology data which does not exist. The first array dimension of that data is a categorical index:

MDTF-diagnostics/diagnostics/AMOC_3D_Structure/sum_MFWT_lats_y_plot.ncl

Lines 69 to 75 in 68fb896

    
           ;==================================================================================================== 
        
           ; k = 0       1       2         3         4         5 
        
           ;  upper, middle0, lower0, middle_min, lower_min, total   
        
           ; i = 0       1       2 
        
           ;   Net, Positive, Negative 
        
           ks=(/0/) 
        
           is=(/0/)

The header data for these files doesn't describe the category values. The submitted code specified the index ks=(/5/) (ie, to use the "total" data) for every climatology file. However, all of the supplied CMIP5 climatology data are arrays of dimension 4 x <# of latitudes> x 3, ie the 5th row was absent, which raised index-out-of-range errors. Attempting to use an index of 0 or 3 eliminates errors, but does not produce plots that appear correct.

4. Legends in summary plots. The legend caption "high-res. model" was changed to "HYCOM" in the following plots in order to clarify the origin of that data: sum_Twgt_lats_y_plot, sum_Swgt_lats_y_plot, sum_MHT_lats_y_plot, sum_MFWT_lats_y_plot [multiple-line plots]; Twgt_vs_Swgt_lat0_dots_plot; MFWT_vs_AMOCr_lat0_dots, MHT_vs_AMOCr_lat0_dots, MFWT_vs_Sdiff_lat0_dots, MHT_vs_Tdiff_lat0_dots, ref_MFWT_vs_AMOCr_lat0_dots, ref_MHT_vs_AMOCr_lat0_dots, ref_MFWT_vs_Sdiff_lat0_dots, ref_MHT_vs_Tdiff_lat0_dots [scatter plots].

5. Including model in scatter plots. In the last eight plots listed above, an extra symbol representing data for the model being analyzed has been added (in addition to the data for the CMIP5 ensemble members and HYCOM).

6. Miscellaneous. Some axes and legend labels are cut off. Many NCL warnings are logged (in particular AutoComputeMajorTickMarks: The precision specified is smaller than the precision needed tick mark labels may not be correct, indicative of an incorrectly set range), which may or may not indicate a problem with the plot being generated. All plots should be checked for correctness and scripts should be fixed accordingly.

0 replies

bitterbark · 2020-09-16T15:13:46Z

bitterbark
Sep 16, 2020
Collaborator

The missing climatological data mentioned in (1) is posted on the ftp site. I remade it with the hard-dereference flag which may have been limiting the data extraction: ftp://ftp.cgd.ucar.edu/archive/mdtf/obs_data.AMOC.20200916.tar

…

On Mon, Sep 14, 2020 at 11:38 AM Thomas Jackson ***@***.***> wrote: The following issues remain and can't be addressed without input from the POD authors: *1. CMIP5 climatology data.* Climatology data was included for 20 models in the CMIP5 ensemble, but some summary plots make reference to data that wasn't provided. It's unclear whether this data was intended to be computed by the POD, or is missing. Included files are *.AMOC_qts.clim.nc, *. MFWT.clim.nc, *.MHT.clim.nc, *.so_wgt.clim.nc, *.thetao_wgt.clim.nc; files which may be missing are *.AMOC_T2B.clim.nc, *.trans_avg.clim.nc, *. lat_vo.fx.nc. *2. Malformed scatter plot.* The plot produced by Twgt_vs_Swgt_lat0_dots_plot.ncl <https://github.com/NOAA-GFDL/MDTF-diagnostics/blob/68fb896a196d68349e4277d7cae387c06222b719/diagnostics/AMOC_3D_Structure/Twgt_vs_Swgt_lat0_dots_plot.ncl> appears incorrect: data is plotted twice on the same set of axes. The source code has many lines commented out which indicate that it may have been intended to be a multi-panel plot, but it's unclear what was intended. *3. Nonexistent rows in climatology data.* sum_MFWT_lats_y_plot.ncl <https://github.com/NOAA-GFDL/MDTF-diagnostics/blob/68fb896a196d68349e4277d7cae387c06222b719/diagnostics/AMOC_3D_Structure/sum_MFWT_lats_y_plot.ncl> and sum_MHT_lats_y_plot.ncl <https://github.com/NOAA-GFDL/MDTF-diagnostics/blob/68fb896a196d68349e4277d7cae387c06222b719/diagnostics/AMOC_3D_Structure/sum_MHT_lats_y_plot.ncl> reference a row in the CMIP5 climatology data which does not exist. The first array dimension of that data is a categorical index: https://github.com/NOAA-GFDL/MDTF-diagnostics/blob/68fb896a196d68349e4277d7cae387c06222b719/diagnostics/AMOC_3D_Structure/sum_MFWT_lats_y_plot.ncl#L69-L75 The header data for these files doesn't describe the category values. The submitted code specified the index ks=(/5/) (ie, to use the "total" data) for every climatology file. However, all of the supplied CMIP5 climatology data are arrays of dimension 4 x <# of latitudes> x 3, ie the 5th row was absent, which raised index-out-of-range errors. Attempting to use an index of 0 or 3 eliminates errors, but does not produce plots that appear correct. *4. Legends in summary plots.* The legend caption "high-res. model" was changed to "HYCOM" in the following plots in order to clarify the origin of that data: sum_Twgt_lats_y_plot, sum_Swgt_lats_y_plot, sum_MHT_lats_y_plot, sum_MFWT_lats_y_plot [multiple-line plots]; Twgt_vs_Swgt_lat0_dots_plot; MFWT_vs_AMOCr_lat0_dots, MHT_vs_AMOCr_lat0_dots, MFWT_vs_Sdiff_lat0_dots, MHT_vs_Tdiff_lat0_dots, ref_MFWT_vs_AMOCr_lat0_dots, ref_MHT_vs_AMOCr_lat0_dots, ref_MFWT_vs_Sdiff_lat0_dots, ref_MHT_vs_Tdiff_lat0_dots [scatter plots]. *5. Including model in scatter plots.* In the last eight plots listed above, an extra symbol representing data for the model being analyzed has been added (in addition to the data for the CMIP5 ensemble members and HYCOM). *6. Miscellaneous.* Some axes and legend labels are cut off. Many NCL warnings are logged (in particular AutoComputeMajorTickMarks: The precision specified is smaller than the precision needed tick mark labels may not be correct, indicative of an incorrectly set range), which may or may not indicate a problem with the plot being generated. All plots should be checked for correctness and scripts should be fixed accordingly. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#20 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHLJVQFCPCUAJBDTUVG3CR3SFZIIFANCNFSM4KXKNGZA> .

0 replies

tsjackson-noaa · 2020-09-16T23:02:22Z

tsjackson-noaa
Sep 16, 2020
Author

Thanks @bitterbark -- did you upload the correct file, though? I'm looking at obs_data.AMOC.20200916.tar and it seems identical to obs_data.AMOC.tar -- in particular, in /clim there are only the five data files for each CMIP5 model (*.AMOC_qts.clim.nc, *.MFWT.clim.nc, *.MHT.clim.nc, *.so_wgt.clim.nc, *.thetao_wgt.clim.nc).

0 replies

bitterbark · 2020-09-16T23:35:55Z

bitterbark
Sep 16, 2020
Collaborator

You're right, it is identical (except for the hard-dereference flag). I misunderstood and that you didn't have any of the clim/ files. As for the files you mentioned that are not included, the tar file includes those for the HYCOM model. % tar tf obs_data.AMOC_3D_Structure.tar | grep -e AMOC_T2B.clim.nc -e trans_avg.clim.nc -e lat_vo.fx.nc AMOC_3D_Structure/fx/HYCOM.lat_vo.fx.nc AMOC_3D_Structure/clim/HYCOM.trans_avg.clim.nc AMOC_3D_Structure/clim/HYCOM.AMOC_T2B.clim.nc But now I understand that you are looking for those for the other models as well. In that case, I guess I can't help!

…

On Wed, Sep 16, 2020 at 5:02 PM Thomas Jackson ***@***.***> wrote: Thanks @bitterbark <https://github.com/bitterbark> -- did you upload the correct file, though? I'm looking at obs_data.AMOC.20200916.tar and it seems identical to obs_data.AMOC.tar -- in particular, in /clim there are only the five data files for each CMIP5 model (*.AMOC_qts.clim.nc, *. MFWT.clim.nc, *.MHT.clim.nc, *.so_wgt.clim.nc, *.thetao_wgt.clim.nc). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#20 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHLJVQAPYV3YWDBFD7QZAPLSGE7YXANCNFSM4KXKNGZA> .

0 replies

tsjackson-noaa · 2021-01-23T20:58:01Z

tsjackson-noaa
Jan 23, 2021
Author

The branch containing existing development work on this POD has been renamed from pod/AMOC_3D to feature/add_AMOC_3D, as part of the reorganization proposed in issue #106.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

POD discussion: AMOC 3D structure diagnostic #118

{{title}}

Replies: 8 comments

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

POD discussion: AMOC 3D structure diagnostic #118

tsjackson-noaa Feb 18, 2020

Replies: 8 comments

tsjackson-noaa Aug 14, 2020 Author

tsjackson-noaa Sep 13, 2020 Author

tsjackson-noaa Sep 13, 2020 Author

tsjackson-noaa Sep 14, 2020 Author

bitterbark Sep 16, 2020 Collaborator

tsjackson-noaa Sep 16, 2020 Author

bitterbark Sep 16, 2020 Collaborator

tsjackson-noaa Jan 23, 2021 Author

tsjackson-noaa
Feb 18, 2020

tsjackson-noaa
Aug 14, 2020
Author

tsjackson-noaa
Sep 13, 2020
Author

tsjackson-noaa
Sep 13, 2020
Author

tsjackson-noaa
Sep 14, 2020
Author

bitterbark
Sep 16, 2020
Collaborator

tsjackson-noaa
Sep 16, 2020
Author

bitterbark
Sep 16, 2020
Collaborator

tsjackson-noaa
Jan 23, 2021
Author