Skip to content

UFrame Web Services API

John Kerfoot edited this page Jan 15, 2016 · 4 revisions

WikiUFrame Web Services

#Contents

#Background

###Reference Designators The Ocean Observatories Initiative uses a reference designator to identify a particular instrument that has been deployed as part of a site. The format of a reference designator is:

SITE-SUBSITE-INSTRUMENT

Here's an example of an actual reference designator:

CP02PMCI-WFP01-03-CTDPFK000

and is translated as:

  • CP02PMCI: Coastal Pioneer Central Inshore Profiling Mooring
  • WFP01: Wire Following Profiler Number 1
  • 03-CTDPFK000: Conductivity, Temperature, Depth Instrument

##Contents

###UFrame Url Configuration

####UFrame Url and Port The UFrame data download request API is located on port 12576 and located under /sensor/inv:

http://$UFRAME_URL:12576/sensor/inv

For the purposes of this discussion, this url will heretofor be referred to as $UFRAME_BASE_URL.

All API responses are provided as valid JSON objects, with the exception of the actual data response, which may also be returned as a NetCDF file.

Pointing a browser to this url results in a json response listing the sites for which instrument data has been ingested into the system. This is the starting point for formulating both a synchronous and asynchronous data download request. It's the top of a data hierarchy that maps as follows:

/SITE
     /SUBSITE
             /INSTRUMENT
                        /TELEMETRY
                                  /STREAM

Understanding this structure is the key to creating valid requests. Definitions and examples for the traversing this hierarchy are provided in the next sections.

####Site

SITE is the name of the site contained under the deployment site for which data is desired. The list of deployment sites can be found here.

The complete list of sites can be found by sending the following request:

$UFRAME_BASE_URL

For which the response is a json array, containing all available sites, looks like this:

[
    "CP02PMCI",
    "GA02HYPM",
    "GP02HYPM",
    "GP05MOAS",
    "CE02SHBP",
    "RS03ECAL",
    "CP04OSPM",
    "CE01ISSM",
    "GS03FLMB",
    "RS03AXPS",
    "GI03FLMA",
    "CP02PMCO",
    "GS03FLMA",
    "GI01SUMO",
    "GA05MOAS",
    "RS03AXBS",
    "RS03INT1",
    "CE05MOAS",
    "RS03INT2",
    "GI05MOAS",
    "GI02HYPM",
    "GA03FLMA",
    "RS03ASHS",
    "GS05MOAS",
    "CE04OSPS",
    "GP03FLMB",
    "GP03FLMA",
    "GS01SUMO",
    "CP05MOAS",
    "RS01SUM3",
    "RS01SUM2",
    "CE04OSBP",
    "GA03FLMB",
    "CP03ISSP",
    "RS01SUM1",
    "CE09OSPM",
    "GI03FLMB",
    "RS01SBPS",
    "CP02PMUI",
    "RS01SLBS",
    "CP01CNSM",
    "CP02PMUO",
    "RS03CCAL",
    "GA01SUMO"
]

####SUBSITE

Once a SITE has been selected, the site id is appended to the $UFRAME_BASE:

$UFRAME_BASE_URL/SITE

and returns a list of all available sub-sites, where a sub-site is the name of the sub-site under the site for which data is desired.

As an example, we'll use the CP05MOAS SITE, which are the Coastal Pioneer Profiling Gliders:

$UFRAME_BASE_URL/CP05MOAS

For which the response is a json array, containing all available sub-sites under the specified SITE, looks like this:

[
    "GL374",
    "GL336",
    "GL335",
    "GL389",
    "GL376",
    "GL388",
    "GL387",
    "GL339",
    "GL379",
    "GL340",
    "GL380"
]

This array contains all of the gliders deployed under the CP05MOAS site.

####INSTRUMENT

Next, we'll select a SUBSITE in order to get a list of the available instruments on the sub-site at the time of deployment. Let's use "GL380" and append this sub-site to our request:

$UFRAME_BASE_URL/SITE/SUBSITE

which looks like this:

$UFRAME_BASE_URL/CP05MOAS/GL380

For which the response is a json array, containing all available instruments on the specified SUBSITE, looks like this:

[
    "04-DOSTAM000",
    "01-ADCPAM001",
    "00-ENG000000",
    "05-PARADM000",
    "03-CTDGVM000",
    "02-FLORTM000"
]

This array contains all of the instruments installed on glider CP380.

####TELEMETRY

Next, we'll select an instrument for the specified subsite in order to see the list of data telemetry options. The term telemetry refers to the method by which the data was sent from the sub site to shore prior to ingestion. Let's look the CTD instrument:

$UFRAME_BASE_URL/SITE/SUBSITE/INSTRUMENT

The url looks like this:

$UFRAME_BASE_URL/CP05MOAS/GP380/03-CTDGVM000

For which the response is a json array, containing all available telemetry methods for the specified INSTRUMENT, looks like this:

[
    "telemetered",
    "recovered_host"
]

This array contains the data telemetry methods for the CTD (03-CTDGVM000) located on glider 380.

####STREAM

Next, we'll select a telemetry type to see the list of available streams. A stream contains a collection of parameters/variables. Let's look at the available telemetered data streams:

$UFRAME_BASE_URL/SITE/SUBSITE/INSTRUMENT/TELEMETRY

The url looks like this:

$UFRAME_BASE_URL/CP05MOAS/GP380/03-CTDGVM000/telemetered

For which the response is a json array, containing all available telemetered data streams, looks like this:

[
    "ctdgv_m_glider_instrument"
]

This array contains one telemetered data stream.

We're almost ready to discuss sending a synchronous or asynchronous data request, but first we need to append the stream to the request url:

$UFRAME_BASE_URL/SITE/SUBSITE/INSTRUMENT/TELEMETRY/STREAM

The url looks like this:

$UFRAME_BASE_URL/CP05MOAS/GP380/03-CTDGVM000/telemetered/ctdgv_m_glider_instrument

which we'll refer to heretofore as $UFRAME_BASE_DATA_URL.

IMPORTANT: Sending the $UFRAME_BASE_DATA_URL with no parameters will result in an asynchronous request to retrieve ALL data for the stream, which will likely be large, take some time to complete and will be made available via an OPeNDAP endpoint only. See the Detailed API Documentation for parameters that can be used to customize the data request.

#API Documentation

Before diving into query parameters, it's important to point out that the UFrame web services API provides a request that can be used to request metadata for a particular instrument. As with all previous requests defined above, the response is provided as a JSON object, and contains the following properties, which map to arrays:

  • parameters
  • streams

A full metadata response for the above example can be requested via the following url:

$UFRAME_BASE_URL/CP05MOAS/GP380/03-CTDGVM000/metadata

and can be found here, but here is an example of a single element from parameters:

{
    parameters: [
        {
            shape: "SCALAR",
            unsigned: false,
            type: "DOUBLE",
            stream: "ctdgv_m_glider_instrument_recovered",
            units: "seconds since 1900-01-01",
            fillValue: "-9999999",
            pdId: "PD7",
            particleKey: "time"
        },
        ...],
    ...
 }

Each element of the parameters array contains metadata about the child parameter of the stream.

And an example from a single element of the 'times' array:

{
    times: [
        {
            endTime: "2015-01-04T18:35:36.420Z",
            method: "recovered_host",
            count: 3566545,
            stream: "ctdgv_m_glider_instrument_recovered",
            sensor: "CP05MOAS-GL380-03-CTDGVM000",
            beginTime: "2014-10-06T23:58:44.162Z"
        },
        ...
    ]
}

Each element of the times array contains the metadata for a specific stream, including the telemetry type (method) as well as the timestamp of the first and least record/particle in the stream (beginTime and endTime, respectively).

Now that we've looked at the metadata for the instrument, let's take a look at using query parameters to customize the request. The following is a list of query parameters that may be added to $UFRAME_BASE_DATA_URL in order to customize the requested data product:

If any of the above parameters are specified, they be specified by appending a question mark (?) to the $UFRAME_BASE_DATA_URL and then appending the parameter. Subsequent parameters are appended using an ampersand (&) followed by the parameter.

##beginDT

The beginDT parameter accepts a ISO8601-formatted combined date/time of the following format:

YYYY-MM-DDTHH:mm:ss.sssZ

and specifies the minimum start time for the data request. Here's an example:

$UFRAME_BASE_DATA_URL?beginDT=2014-10-06T23:58:44.162Z 

##endDT

The endDT parameter accepts a ISO8601-formatted combined date/time of the following format:

YYYY-MM-DDTHH:mm:ss.sssZ

and specifies the maximum end time for the data request. Here's an example (appended the the beginDt parameter):

$UFRAME_BASE_DATA_URL?beginDT=2014-10-06T23:58:44.162Z&endDT=2015-01-04T18:35:36.420Z

##format

If not specified, the resulting response is returned as a JSON data object. Valid options for this parameter are:

  • application/netcdf: NetCDF file
  • application/json: JSON object
  • application/csv: Comma-separated value file
  • application/tsv: Tab-separated value file

##execDPA

If not specified, only Level 0 parameters are returned. If set to true, i.e.:

&execDPA=true

Level 1 and level 2 (derived) parameters are also returned. You'll likely want to set this option to true. If we go with the previous URL, the request would look like this:

$UFRAME_BASE_DATA_URL?beginDT=2014-10-06T23:58:44.162Z&endDT=2015-01-04T18:35:36.420Z&execDPA=true

##limit

##include_provenance #Examples

Clone this wiki locally