Merge pull request #527 from wrongkindofdoctor/refactor_pp

clean up dev docs and copy_external_docs

doc/copy_external_docs.py

@@ -68,7 +68,7 @@ def _docname(item):
         os.makedirs(sphinx_dir)
 
     # find PODs or sites as directories under search_root
-    entries = [x for x in os.listdir(search_root) \
+    entries = [x for x in os.listdir(search_root)
                if os.path.isdir(os.path.join(search_root, x)) and x[0].isalnum()
                ]
     # Case-insensitive alpha sort
@@ -77,10 +77,6 @@ def _docname(item):
     if 'example' in entries:
         entries.remove('example')
         entries.insert(0, 'example')
-    # put local site documentation first
-    elif 'local' in entries:
-        entries.remove('local')
-        entries.insert(0, 'local')
 
     # find documentation files
     # = all non-PDF files (.rst and graphics) in /doc subdirectory

doc/index.rst

@@ -21,6 +21,7 @@ This site contains documentation for the MDTF-Diagnostics package. The code is f
    sphinx/ref_toc
    sphinx/fmwk_toc
    sphinx/pod_toc
+   sphinx/tools_toc
 
 
 Acknowledgements

doc/sphinx/dev_cheatsheet.rst

@@ -5,31 +5,45 @@ Creating and submitting a POD
 -----------------------------
 1. Prepare for implementation  
 
-   - Run the unmodified MDTF-diagnostics package to make sure that your conda installation, directory structure, etc... are set up properly  
-   - Modify the conda environment to work for your POD by adding a configuration file ``MDTF_diagnostics/src/conda/env_[YOUR POD NAME].yml`` with any new required modules.  Be sure to re-run ``MDTF-diagnostics/src/conda/conda_env_setup.sh`` to install your POD's environment if it requires a separate YAML file with additional modules.
-   - Name your POD, make a directory for your POD in MDTF-diagnostics/diagnostics, and move your code to your POD directory  
+   - Run the unmodified MDTF-diagnostics package to make sure that your conda installation, directory structure, etc...
+     are set up properly
+   - Modify the conda environment to work for your POD by adding a configuration file
+     ``MDTF_diagnostics/src/conda/env_[YOUR POD NAME].yml`` with any new required modules.  Be sure to re-run
+     ``MDTF-diagnostics/src/conda/conda_env_setup.sh`` to install your POD's environment if it requires a separate YAML
+     file with additional modules.
+   - Name your POD, make a directory for your POD in MDTF-diagnostics/diagnostics, and move your code to your POD
+     directory
    - ``cp`` your observational data to ``MDTF_diagnostics/../inputdata/obs_data/[YOUR POD NAME]``
 2. Link your POD code into the framework
    - Modify your POD's driver script (e.g, ``driver.py``) to interface with your code
    - Modify pod's ``settings.jsonc`` to specify variables that will be passed to the framework
-   - Modify your code to use ``ENV_VARS`` provided by the framework (see the *Notes* for descriptions of the available environment variables)
+   - Modify your code to use ``ENV_VARS`` provided by the framework (see the *Notes* for descriptions of the available
+     environment variables)
       - Input files:
          - model input data: specified in an ESM-intake catalog
          - observational input data: ``MDTF-diagnostics/../inputdata/obs_data/[POD name]``
-         - You may re-define input data locations in the ``OBS_DATA_ROOT`` setting in your runtime configuration file (or whatever the name of your runtime settings jsonc file is).
+         - You may re-define input data locations in the ``OBS_DATA_ROOT`` setting in your runtime configuration file
+           (or whatever the name of your runtime settings jsonc file is).
       - Working files: 
-         - ``${WORK_DIR}`` is a framework environment variable defining the working directory. It is set to ``MDTF-diagnostics/../wkdir`` by default.
+         - ``${WORK_DIR}`` is a framework environment variable defining the working directory. It is set to
+         ``MDTF-diagnostics/../wkdir`` by default.
          - ``${WORK_DIR}`` contains temporary files and logs.
-         - You can modify ``${WORK_DIR}`` by changing "WORK_DIR" to the desired location in ``templates/runtime.[jsonc |yml}``
+         - You can modify ``${WORK_DIR}`` by changing "WORK_DIR" to the desired location in
+           ``templates/runtime.[jsonc |yml}``
       - Output files: 
          - POD output files are written to the following locations by the framework:
-            - Postscript files: ``${WORK_DIR}/[POD NAME]/[model,obs]/PS``
-            - Other files, including PNG plots: ``${WORK_DIR}/[POD NAME]/[model,obs]``
-         - Set the "OUTPUT_DIR" option in default_tests.jsonc to write output files to a different location; "OUTPUT_DIR" defaults to "WORK_DIR" if it is not defined.
+            - Postscript files: ``${WORK_DIR}/MDTF_output[.v#]/[POD NAME]/[model,obs]/PS``
+            - Other files, including PNG plots: ``${WORK_DIR}/MDTF_output[.v#]/[POD NAME]/[model,obs]``
+         - Set the "OUTPUT_DIR" option in default_tests.jsonc to write output files to a different location;
+           "OUTPUT_DIR" defaults to "WORK_DIR" if it is not defined.
          - Output figure locations:  
             - PNG files should be placed directly in ``$WORK_DIR/obs/`` and ``$WORK_DIR/model/``
-            - If a POD chooses to save vector-format figures, they should be written into the ``$WORK_DIR/obs/PS`` and ``$WORK_DIR/model/PS`` directories. Files in these locations will be converted by the framework to PNG, so use those names in the html file.
-            - If a POD uses matplotlib, it is recommended to write as figures as EPS instead of PS because of potential bugs
+            - If a POD chooses to save vector-format figures, they should be written into the
+              ``$WORK_DIR/MDTF_output[.v#]/[POD_NAME]/obs/PS`` and
+              ``$WORK_DIR/MDTF_output[.v#]/[POD_NAME]/model/PS`` directories. Files in these locations will be
+              converted by the framework to PNG, so use those names in the html file.
+            - If a POD uses matplotlib, it is recommended to write as figures as EPS instead of PS because of potential
+              bugs
 
    - Modify html files to point to the figure names
 
@@ -39,22 +53,33 @@ Creating and submitting a POD
 
 Notes:
 ------
-- **Make sure that WORK_DIR and OUTPUT_DIR have enough space to hold data for your POD(s) AND any PODs included in the package.**
+- **Make sure that WORK_DIR and OUTPUT_DIR have enough space to hold data for your POD(s) AND any PODs included in the
+  package.**
 - Defining POD variables
-   - Add variables to the ``varlist`` block in the ``MDTF-diagnostics/diagnostics/[POD name]/settings.jsonc`` and define the following:
-      - the variable name: the short name that will generate the corresponding ``${ENV_VAR}`` (e.g., "zg500" generates the ``${ENV_VAR}`` "zg500_var")
+   - Add variables to the ``varlist`` block in the ``MDTF-diagnostics/diagnostics/[POD name]/settings.jsonc`` and define
+     the following:
+      - the variable name: the short name that will generate the corresponding ``${ENV_VAR}``
+        (e.g., "zg500" generates the ``${ENV_VAR}`` "zg500_var")
       - the standard name with a corresponding entry in the appropriate fieldlist file(s)  
       - variable units
       - variable dimensions (e.g., [time, lat, lon])
       - variable realm (e.g., atmos, ocean ice, land)
-      - scalar coordinates for variables defined on a specific atmospheric pressure level (e.g. ``{"lev": 250}`` for a field on the 250-hPa p level).
+      - scalar coordinates for variables defined on a specific atmospheric pressure level (e.g. ``{"lev": 250}``
+        for a field on the 250-hPa p level).
 
-   - If your variable is not in the necessary fieldlist file(s), add them to the file(s), or open an issue on GitHub requesting that the framework team add them. Once the files are updated, merge the changes from the main branch into your POD branch.
+   - If your variable is not in the necessary fieldlist file(s), add them to the file(s), or open an issue on GitHub
+     requesting that the framework team add them. Once the files are updated, merge the changes from the main branch
+     into your POD branch.
    - Note that the variable name and the standard name must be unique fieldlist entries
 
 - Environment variables
-   - To define an environment variable specific to your POD, add a ``"pod_env_vars"`` block to the ``"settings"`` block in your POD's ``settings.jsonc`` file and define the desired variables
-   - Reference an environment variable associated with a specific case in Python by calling ``os.environ[case_env_file]``, reading the file contents into a Python dictionary, and getting value associated with the first case (assuming variable names and coordinates are identical for each case), e.g. ``tas_var = [case['tas_var'] for case in case_list.values()][0]``. See ``example_multicase.py`` for more information.
+   - To define an environment variable specific to your POD, add a ``"pod_env_vars"`` block to the ``"settings"``
+     block in your POD's ``settings.jsonc`` file and define the desired variables
+   - Reference an environment variable associated with a specific case in Python by calling
+     ``os.environ[case_env_file]``, reading the file contents into a Python dictionary, and getting value associated
+     with the first case (assuming variable names and coordinates are identical for each case), e.g.
+     ``tas_var = [case['tas_var'] for case in case_list.values()][0]``. See ``example_multicase.py`` for more
+     information.
    - NCL code can reference environment variables by calling ``getenv("VARIABLE NAME")``  
    - Framework-specific environment variables include:
       - case_env_file: path to yaml file with case-specific environment variables: