Run & examine a basic HARP point verification for a DE_330 test case (from vfld vobs files)

• After checking that the HARP installation in your user account at ATOS / other system is correct, the next step is to produce a basic point verification for your experiment. In this tutorial, one of the test cases delivered by task 3.1 from WP5 will be used as an example: https://github.com/DEODE-NWP/WP53/wiki/Case-studies-associated-with-M330531 .

• First, select one of the case studies included in dcmdb. We will refer to every case study by the combination "Region+Date" from the table in the link above. These instructions assume that you have access to:

  a) Observation files provided by the case study producers. These are operational vobs files or OBSTABLE*.sqlite files. Currently a collection of these for many of the case studies are available at ATOS in these paths: /ec/res4/scratch/sp3c/deode_verif/cases/{Region_date}/vobs/ and /perm/sp3c/deode_verif/cases/{Region_date}/OBSTABLE/ for

  b) the vfld files extracted from the forecasts (for harmonie cases), or the model output files of the experiments in grib2/FA format (for interpolating to the station list). These will be converted to FCTABLEsqlite files by one of the steps in the verification process. For the reference runs, there's already a collection of FCTABLEsqlite files in path /perm/sp3c/deode_verif/cases/{Region_date}/FCTABLE/{Reference_experiment_name}/{YYYY}/{MM}

• Secondly, download the DEODE-NWP/deode_harp_verification_scripts which have been adapted from harphub/accord_verification_scripts. For a minimal modification of the existing scripts/configuration files for your case, it is suggested to do it in a folder called "deode_verif" in your $PERM:

cd $PERM; mkdir deode_verif; cd deode_verif; git clone https://github.com/DEODE-NWP/DE_330-verif-scripts.git .

• Don't forget the "." in the last command. This way "deode_verif" will be populated with the scripts from the git repository. You will find in $PERM/deode_verif/main.sh the main script where you specify what steps of the verification process to run, what configuration yml file to load, etc.

• Go to $PERM/deode_verif/config and check if a config_atos.yml_{Region_Date} exists for your case study. Substitute all paths in the file with your own paths to vfld/vobs files model output files, etc, the experiment names with the ones from your test case, the corresponding start_date and end_date, forecast lengths (fclen), frequency of observations in vobs (by_vobs_step), time between model runs (by_step), etc. If you just want to test that your harp installation works with the example in the script, you can keep vfld_path and vobs_path untouched in the config file (check that they're still available at the paths). Create the main folder where the files for your case study will be stored $PERM/deode_verif/cases/Region_Date.

• Now you're ready to run the main verification script, main.sh (currently it's a link to scr/master_atos.sh, but you could adapt this script if running in a different system). First, edit the file (modify export CASE_STUDY={Region_Date}) indicating the name of your case study so that the script can read the correct config file (config_atmos.yml_{Region_Date}). The script main.sh calls to different R scripts; each one will complete a different step of the verification process, each with a dedicated variable in main.sh which can be set to yes/no to activate/deactivate every step. This is because some steps need only to be run once (i.e. conversion of vobs to FCTABLE*sqlites, etc). So one good idea (specially the first time) is to run the needed steps in main.sh one by one. This saves time when debugging if something goes wrong. The steps are these:

  1. Convert vobs to sql. Activate with RUN_VOBS2SQL=yes. Only for cases where vobs files are available (mainly Harmonie cases). Otherwise, check that OBSTABLES* exist for your case study at /perm/sp3c/deode_verif/cases/{Region_date}/OBSTABLE/ or provide them separately.
  2. Convert vfld to sql. Activate with RUN_VFLD2SQL=yes. Only for cases where vfld files are available (mainly Harmonie cases). Otherwise, check that FCTABLES* exist for your case study / experiment at /perm/sp3c/deode_verif/cases/{Region_date}/FCTABLE/{Reference_experiment_name}/{YYYY}/{MM} or provide them.
  3. Interpolate model output to sql. Alternative to point 2 for cases with no vfld files available. The model output files (grb/FA files) must be available. Activate with RUN_INTERPOL2SQL=yes
  4. Run point verification for variables defined in config/config_atos.R. Activate with RUN_POINT_VERF=yes
  5. Run complete verification set, figures, etc. Activate with RUN_POINT_VERF_LOCAL=yes
  6. Generate scorecards (experimental). They're stored as a .png file in the /output directory. Activate with SCORECARDS=yes
  7. Open dynamic or static shiny web page to inspect results. Select one port number in SHINY_PORT variable and activate web page launch with SHOW_WEB_DYNAMIC=yes or SHOW_WEB_STATIC=yes ( See instructions printed in screen to open the web app in a web browser at Bolonia's virtual desktop).

• After running all steps, if everything went fine, you will have the following folders in your $PERM/deode_verif/cases/RegionDate folder:

↳ $PERM/deode_verif/cases/Region_Date
     ↳ FCTABLE where the vlfd conversions to sqlite are stored for every experiment
     ↳ OBSTABLE where the vobs conversions to sqlite are stored for every experiment
     ↳ output
       ↳ verif_results. Where the .rds R data files with verification results created in step 3 are stored. These are used later to feed the interactive visualization web page.
       ↳ start_date-end_date. Where a big number of .png figures with verification results created in step 4 are stored, including maps showing station bias, etc...which are not presented in the visualization web page.

• Dynamic page: After opening the verification page, look for "Select Verification Directory" and navigate to "$PERM/deode_verif/cases/RegionDate/verif_results" to start visualizing your verification results.

• Static page: After opening the verification page, select the case study in the top left menu. The rest of the dropdown menus will update according to available pre-generatred figures