data.get

data.get serves as the primary means of retrieving data from the archives. It can be used to get data from any source over any given time range. This can include processing other data to generate “new” data (e.x. edited data).

data.get also caches retrieved data from sources it is configured to cache. It will use the cached data over any archive fetch.

Output can either by to standard out or to a file, who's name is generated automatically. If writing to a file the target is erased if it already exists.

Uses get.sources.conf to construct available data sources and caching.conf to determine what is cached.

Command Line Usage

data.get [--stdout] [--file] [--timestamp] 
         [--localdata] [--nocache] [--cpx]
         [station] records start [end] [source]

The default is to direct output to standard output. If --file is set output is directed to a file named '{record}.{type}'. The default type is 'raw' and the station is attempted to be inferred from the working directory and the environment variable $STATION or $MYSTN if it is absent.

Arguments

start and end

The time specifiers for the data to be retrieved. Start is inclusive while end is exclusive, so all data contained within the half open interval [start,end) will be returned. Any convertible time format is accepted.

station

The station identifier code. For example 'brw'. Case insensitive.

records

The cpd2 record type to be retrieved. For example: 'S11a'. Case sensitive. Multiple record types may be separated by “,”, “;” or “:”. The results will be interwoven in time. Note that this is a single argument and that spaces are not allowed.

source

Select the source archive to request data from, defaulting to raw.

--stdout

If present output is directed to standard out, instead of the target file. No target file is created or altered. This is the default mode.

--file

If present output is directed to a generated file instead of standard output.

--timestamp

If present the output file is changed to '{record}_yyyy-mm-ddThh:mm:ssZ.{type}'. The timestamp is what is given for the start. Note that --stdout overrides this.

--localdata

If present the local station archive is also searched for data. This is use to retrieve local station data on that station's computer. It is used by cpx2 for station operation. It is normally meaningless for data types other than 'raw'. The exact function of this argument is to include any source lines tagged with the argument 'localdata' that are normally excluded.

--nocache

Disable caching. This disables both reading from the cache and writing to it.

--cpx

CPX2 mode. Abort on any input from standard input.

Example Usage

Single record

data.get sgp S11a 2008:10 2008:11

Multiple records of clean data

data.get bnd S11a,A11a 2003W02 2003W03 clean

Writing a file

data.get --file sgp A11m 2008.0 2008.25

Information

Data