publish.check.conf

This file defines the sanity checks done by data.aggregate.publishnotify. Any checks that fail cause a note to be added to the email sent out on completion of passing a quarter. It is located in $DB/etc/$STATION/publish.check.$STATION.conf or in the default location $DB/etc/publish.check.conf. If the station specific file does not exist then the default one is used.

Lines beginning with '#' are treated as comments. The format consists of a comma separated list of values where the first value is the input specifier, the second is the type of check and any further values are parameters to that check. The type of check is case insensitive. All data is retrieved from the avgH data source (so it has cut size splitting, standard deviation, and point counts available).

The input specification field consists of a semicolon separated list of inputs. The usage of the inputs depends on the type of check (see below). Each individual input can either be a special processing type or a variable specification. Variable specifications can be full Perl regular expressions that are wrapped like “/^$variable$/”. If the input does specify a special processing type than the regular expression can expand to multiple variables and the rule would be run on all of them (assuming the check type supports that). The result of this is for simple check types processing single simple variables only a single regular expression is needed. That is the rule “Bs[BGRF][01]_S11,Change” checks all neph scatterings for change between hours, but “Fraction:1.0::BbsB0_S11:BsB0_S11,Min,0.04” (the minimum backscatter check) must be spelled out for each color channel, as the “Fraction” processing does not support multiple resolution.

Processing types are case insensitive and have their parameters delimited by “:”.

Calculates a fraction of two inputs.

The first and second parameters are the required minimum and maximum values, respectively, for both inputs to be within. If either value exceeds the limit then the result is a missing value (generally causing the associated check not to be run). If either limit is blank then it is not enforced. The third and fourth parameters are the variable specifiers of the numerator and denominator of the fraction, respectively. If they resolve to multiple values then the first valid (non-missing) one is used.

Calculates the angstrom exponent of two inputs.

The first and second parameters are the required minimum and maximum values, respectively, for both inputs to be within. If either value exceeds the limit then the result is a missing value (generally causing the associated check not to be run).

If there are six parameters total then the third parameter is the wavelength in nm of the longer wavelength channel, while the fourth parameter is the wavelength of the shorter wavelength channel.

If there are five parameters total then the third parameter is the wavelength in nm of the shorter channel, while the longer wavelength one defaults to 700 nm.

If there are four parameters total then the first shorter wavelength defaults to 550 nm and the longer one to 700 nm.

The final two parameters are the shorter and longer wavelength variable specifiers, respectively. If they resolve to multiple values then the first valid (non-missing) one is used.

Calculates single scattering albedo.

The first and second parameters are the required minimum and maximum values, respectively, that scattering must be within. If either value exceeds the limit then the result is a missing value (generally causing the associated check not to be run). If either limit is blank then it is not enforced. The third and fourth parameters are the variable specifiers of the scattering and absorption, respectively. If they resolve to multiple values then the first valid (non-missing) one is used.

Each rule has a check type that is run on its inputs. Some check types support running on multiple inputs unordered (“Min” for example), while others require their inputs in a certain order (“Cascade” for example). The parameters to each are the remaining comma separated values in the rule definition. If a check fails it generates a note in the email report.

This is a special check type that takes no inputs. All of its parameters are added to the set of records to load. This set should contain all the variables used in any rules.

Fails when its input does not change from one record to the next. Can accept multiple inputs independently.

Fails when its input drops below the value of its first parameters. Can accept multiple inputs independently.

Fails when its input exceeds the value of its first parameters. Can accept multiple inputs independently.

Always succeeds when either of its inputs is less than the value of its second parameter (the minimum applicable threshold). Fails when the low (second) input is greater than the value of the first parameter (threshold) times the value of the high (first) input. That is it enforces “low < high * threshold”. Accepts two ordered inputs: the high and low values, respectively.