Merge branch 'latest'

* latest: (1367 commits)
  Move vicinity processing to threshold CLI (metoppv#581)
  Refactored to allow for separate filling functions (metoppv#587)
  Add warning if too many zero values for advection velocities (metoppv#585)
  Impro 621 improve snow (metoppv#586)
  metoppv#442: Add profiling support to CLIs (metoppv#554)
  Update readthedocs config to include netcdftime
  Add test for licence information (metoppv#582)
  IMPRO-639: GenerateProbabilitiesFromMeanAndVariance plugin (metoppv#578)
  Removal of renaming of coordinates within ensemble calibration, as this functionality is not used, as no input is provided with ensemble_member_id or ensemble_realization_id as a coordinate. (metoppv#579)
  Add coverage badge to README
  Try out coverage (metoppv#565)
  Impro-672 use realizations rather than members (metoppv#577)
  Modifications to allow neighbourhood processing of complex numbers (metoppv#570)
  Add support for providing a verbose option for the unit tests. (metoppv#569)
  IMPRO-618: Grid to grid interpolation (metoppv#566)
  IMPRO-212: A faster percentile method. (metoppv#568)
  Impro 624 add coordinate collapse (metoppv#567)
  Add Python version to README.md
  Port IMPROVER to Python 3 (metoppv#558) (metoppv#564)
  Reordered KGO creation call position so it gets made even when test fails. (metoppv#562)
  ...

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,4 +1,6 @@
-Reference issue if exists
+Addresses #GitHubissuenum
+
+Description
 
 Testing:
  - [ ] Ran tests and they passed OK

.gitignore

@@ -14,3 +14,10 @@
 \#*
 \.\#*
 *.swp
+
+# Site-specific setup
+etc/site-init
+
+# Files from build of doc
+*.rst
+/doc/build/

.pylintrc

@@ -0,0 +1 @@
+etc/pylintrc

.readthedocs.yml

@@ -0,0 +1,2 @@
+conda:
+    file: environment.yml

.travis.yml

@@ -4,7 +4,7 @@
 
 language: python
 python:
-  - 2.7
+  - 3.6.5
 sudo: false
 
 git:
@@ -29,43 +29,16 @@ install:
   - conda create --quiet -n $ENV_NAME python=$TRAVIS_PYTHON_VERSION
   - source activate $ENV_NAME
 
-  # Download Iris 1.12.
-  - wget https://github.com/SciTools/iris/archive/v1.12.0.tar.gz -O $HOME/iris.tar.gz
-  - tar -xvf $HOME/iris.tar.gz -C $HOME
-  - cd $HOME/iris-1.12.0
+  # Download Iris 2.0 and all dependencies.
+  - conda install -c conda-forge iris=2.0
 
-  # Add scitools requirements
-  - conda config --add channels scitools
-  - conda install --quiet --file conda-requirements.txt
-  - PREFIX=$HOME/miniconda/envs/$ENV_NAME
+  # Install our own extra dependencies (+ filelock for Iris test).
+  - conda install -c conda-forge filelock mock netcdf4=1.3.1 numpy=1.13.3 pycodestyle pylint pandas python-stratify sphinx=1.7.0 coverage
+  - pip install codacy-coverage
 
-  # Output debug info
+  # List the name and version of all the dependencies within the conda environment.
   - conda list
-  - conda info -a
-
-  # Setup
-  - python setup.py --with-unpack build_ext --include-dirs=${PREFIX}/include --library-dirs=${PREFIX}/lib
-  - IRIS=$(ls -d1 build/lib*/iris)
-  - mkdir $IRIS/etc
-
-  # Set config paths
-  - SITE_CFG=$IRIS/etc/site.cfg
-  - echo "[System]" > $SITE_CFG
-  - echo "udunits2_path = $PREFIX/lib/libudunits2.so" >> $SITE_CFG
-
-  # Build Iris
-  - python setup.py --quiet --with-unpack build
-  - python setup.py --quiet --with-unpack install
-
-  # Get pylint
-  - pip install pylint
-
-  # CD back to IMPROVER
-  - cd -
 
 script:
   - python -c "import iris"
-  - cd lib
-  - pep8 improver
-  - pylint --extension-pkg-whitelist=numpy -E --rcfile=../etc/pylintrc improver
-  - python -m unittest discover
+  - bin/improver tests

ACKNOWLEDGEMENTS.md

@@ -3,3 +3,14 @@ Credit for external pieces of work:
 Iris (https://github.com/SciTools/iris), LGPL:
   - .travis.yml, derived from Iris travis setup
   - .gitignore used as basis for ours
+  - environment.yml derived from Iris minimal conda environment
+
+BATS (https://github.com/sstephenson/bats), MIT-style:
+  - tests/bin/bats\*, unaltered, from commit 0360811
+
+Sphinx (http://www.sphinx-doc.org), BSD:
+  - doc/Makefile, doc/source/conf.py and doc/source/index.rst derived
+    from defaults generated by `sphinx-quickstart`.
+  - Generation of API documentation during build uses
+    `sphinx-apidoc` and is based on solution posted by BowenFu:
+    https://github.com/rtfd/readthedocs.org/issues/1139

CONTRIBUTING.md

@@ -0,0 +1,102 @@
+# IMPROVER: How to Contribute
+
+
+## Raising Bug Reports or Feature Requests
+
+Please raise issues to capture bugs or feature requests - or feel free
+to contact us directly and discuss things if you have our contact details.
+
+
+## Contributing Code or Reviewing
+
+We really appreciate code or review contributions!
+
+Code contributions are made by raising pull requests against the master
+branch of https://github.com/metoppv/improver/. If you are a new contributor,
+your pull request must include adding your details to the list of contributors
+under the [Code Contributors](#code-contributors) part of this page.
+
+Reviewers of the pull requests must check this has been done before the pull
+request is merged into master.
+
+We have a checklist for making sure code is ready to merge in:
+
+https://github.com/metoppv/improver/wiki/Definition-of-Done
+
+and guidance for going through the review process:
+
+https://github.com/metoppv/improver/wiki/Guidance-for-Reviewing-Code
+
+
+## Code Contributors
+
+The following people have contributed to this code under the terms of
+the Contributor Licence Agreement and Certificate of Origin detailed
+below:
+
+* Paul Abernethy (Met Office, UK)
+* Benjamin Ayliffe (Met Office, UK)
+* Mark Baker (Met Office, UK)
+* Laurence Beard (Met Office, UK)
+* Gavin Evans (Met Office, UK)
+* Ben Fitzpatrick (Met Office, UK)
+* Martina Friedrich (Met Office, UK, pre-GitHub)
+* Aaron Hopkinson (Met Office, UK)
+* Kathryn Howard (Met Office, UK)
+* Simon Jackson (Met Office, UK)
+* Caroline Jones (Met Office, UK)
+* Stephen Moseley (Met Office, UK)
+* Meabh NicGuidhir (Met Office, UK)
+* Tim Pillinger (Met Office, UK)
+* Fiona Rust (Met Office, UK)
+* Caroline Sandford (Met Office, UK)
+* Tomasz Trzeciak (Met Office, UK)
+* Mark Worsfold (Met Office, UK)
+
+(All contributors on GitHub are identifiable with email addresses in the
+version control logs or otherwise.)
+
+
+## Contributor Licence Agreement and Certificate of Origin
+
+By making a contribution to this project, I certify that:
+
+(a) The contribution was created in whole or in part by me and I have
+    the right to submit it, either on my behalf or on behalf of my
+    employer, under the terms and conditions as described by this file;
+    or
+
+(b) The contribution is based upon previous work that, to the best of
+    my knowledge, is covered under an appropriate licence and I have
+    the right or permission from the copyright owner under that licence
+    to submit that work with modifications, whether created in whole or
+    in part by me, under the terms and conditions as described by
+    this file; or
+
+(c) The contribution was provided directly to me by some other person
+    who certified (a) or (b) and I have not modified it.
+
+(d) I understand and agree that this project and the contribution
+    are public and that a record of the contribution (including my
+    name and email address) is maintained for the full term of the copyright
+    and may be redistributed consistent with this project or the licence(s)
+    involved.
+
+(e) I, or my employer, grant to the UK Met Office and all recipients of
+    this software a perpetual, worldwide, non-exclusive, no-charge,
+    royalty-free, irrevocable copyright licence to reproduce, modify,
+    prepare derivative works of, publicly display, publicly perform,
+    sub-licence, and distribute this contribution and such modifications
+    and derivative works consistent with this project or the licence(s)
+    involved or other appropriate open source licence(s) specified by
+    the project and approved by the
+    [Open Source Initiative (OSI)](http://www.opensource.org/).
+
+(f) If I become aware of anything that would make any of the above
+    inaccurate, in any way, I will let the UK Met Office know as soon as
+    I become aware.
+
+(The IMPROVER Contributor Licence Agreement and Certificate of Origin is
+derived almost entirely from the Rose version
+(https://github.com/metomi/rose/), which was inspired by the Certificate of
+Origin used by Enyo and the Linux Kernel.)

LICENSE

@@ -1,6 +1,6 @@
 BSD 3-Clause License
 
-British Crown Copyright 2017 Met Office.
+British Crown Copyright 2017-2018 Met Office.
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

README.md

@@ -2,5 +2,9 @@
 
 [![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)
 [![Build Status](https://travis-ci.org/metoppv/improver.svg?branch=master)](https://travis-ci.org/metoppv/improver)
+[![Codacy Badge](https://api.codacy.com/project/badge/Grade/f7dcb46e8e1b4110b3d194dba03fe526)](https://www.codacy.com/app/metoppv_tech/improver?utm_source=github.com&utm_medium=referral&utm_content=metoppv/improver&utm_campaign=Badge_Grade)
+[![Codacy Badge](https://api.codacy.com/project/badge/Coverage/f7dcb46e8e1b4110b3d194dba03fe526)](https://www.codacy.com/app/metoppv_tech/improver?utm_source=github.com&utm_medium=referral&utm_content=metoppv/improver&utm_campaign=Badge_Coverage)
+[![Documentation Status](https://readthedocs.org/projects/improver/badge/?version=latest)](http://improver.readthedocs.io/en/latest/?badge=latest)
+[![Python 3.6](https://img.shields.io/badge/python-3.6-blue.svg)](https://www.python.org/downloads/release/python-360/)
 
 IMPROVER is a library of algorithms for meteorological post-processing and verification.

bin/improver

@@ -0,0 +1,80 @@
+#!/bin/bash
+#------------------------------------------------------------------------------
+# (C) British Crown Copyright 2017-2018 Met Office.
+#------------------------------------------------------------------------------
+# NAME
+#    improver - IMPROVER post-processing and verification operations
+#
+# SYNOPSIS
+#    improver OPERATION [OPTIONS] [ARGS...] # Invoke an IMPROVER operation
+#    improver help                     # Generic help across operations
+#    improver help OPERATION           # Specific help for a particular operation
+#    improver version                  # Print out version information
+#
+# DESCRIPTION
+#    Launch particular operations for post-processing or verification of
+#    meteorological data. This script is a central launcher for all
+#    IMPROVER subcommands (improver-xxxx) and central help.
+#
+# ENVIRONMENT
+#    IMPROVER_SITE_INIT     # override default location for etc/site-init file
+#------------------------------------------------------------------------------
+
+set -eu
+
+export IMPROVER_DIR="$(cd $(dirname $0)/../ && pwd -P)"
+
+# List all improver subcommands or operations.
+get_operations() {
+    cd "$IMPROVER_DIR/bin/"
+    ls improver-*[^~] | sort | sed "s/^improver-//"
+}
+
+# Print generic or operation specific help.
+print_help() {
+    OPER=${1:-}
+    if [[ -n "$OPER" ]] && [[ $OPER != all ]]; then
+        # Operation-specific help.
+        "$IMPROVER_DIR/bin/improver" "$OPER" --help
+    else
+        # General help.
+        sed -n '/^# NAME/,/^#---/{/^#\-/d; s/^#//; s/^ //; p}' "$0"
+        echo
+        echo "OPERATIONS"
+        get_operations | sed "s/^/   improver /"
+    fi
+}
+
+# Print the version.
+print_version() {
+    cat "$IMPROVER_DIR/etc/VERSION"
+}
+
+
+HELP_TARGET=all
+if (($# == 0)); then
+    print_help
+    exit 0
+fi
+if [[ $1 == help ]] || [[ $1 == --help ]]; then
+    print_help ${2:-}
+    exit 0
+fi
+if [[ $1 == version ]] || [[ $1 == --version ]]; then
+    print_version
+    exit 0
+fi
+
+OPER=$1
+shift
+
+# Apply site-specific setup if necessary.
+if [[ -f "${IMPROVER_SITE_INIT:=$IMPROVER_DIR/etc/site-init}" ]]; then
+    . "$IMPROVER_SITE_INIT"
+fi
+
+# Put our library and scripts in the paths.
+export PYTHONPATH="$IMPROVER_DIR/lib/:${PYTHONPATH:-}"
+export PATH="$IMPROVER_DIR/bin/:$PATH"
+
+exec improver-$OPER "$@"