humidograph.conf
The configuration file used by data.aggregate.humidograph. It is located in $DB/etc/$STATION/humidograph.$STATION.conf or in the default location $DB/etc/humidograph.conf. If the station specific file does not exist then the default one is used.
Format
Lines beginning with '#' are treated as comments. Each line is treated as a set of comma separated values with standard quoting applied. The file is divided into “blocks” that are nested together. A block begins with a specification and ends with a single line containing “END”. All key names are case insensitive.
The top level block is a profile block specifying the name of the profile being modified and the time range the modification is for. All profile blocks for the same profile at the same time are overlaid on top of each other, meaning that later blocks in the file can modify parts of the profile without re-defining all of it. The first field of the profile block is the name of the profile, and subsequent fields (if any) are the time range the modification is effective for.
Dry and Wet blocks
Within the profile block, there are “Dry” and “Wet” blocks defining the dry and wet instruments.
Instrument
The instrument key within the block defines the general instrument (e.x. “S11”) to look up data based on. In most cases setting this is all that is necessary.
Records
All occurrences of the records key are added to the records requested. This is only needed if any variables used are not contained in the “a” record for the instrument.
Variables
This sets variables the fit is done on. Each occurrence is a regular expression and all matching variables are added to the fit candidates. Variables are paired between wet and dry and only those that exist on both are fitted. Wet variables are adjusted the same wavelengths as the dry variables, if applicable.
Td
This key takes two parameters both value specifications (see below). The first is a temperature and the second is a humidity. A dewpoint is calculated using the temperature and humidity then the internal RH of the instrument is recalculated using this dewpoint and its internal temperature.
U
This key takes a single value specification and overrides the internal RH for the instrument.
T
This key takes a single value specification and overrides the internal temperature for the instrument. This is generally only used in conjunction with the dewpoint calculation above.
Fits
This block contains a list of sub-blocks that are identified by an internal name (not visible outside the configuration). Each sub-block contains the specification for an instance of the humidogram fit to run. The common case is one sub-block for the global fit, one for the low RH and one for the high RH.
Description
This is a simple description of the fit being run. It it used to generate the metadata description of the variables produced.
Require and Exclude
These keys (multiple instances allowed) set the conditions that data is filtered on before entering into the fit. All “require” conditions must be met for data to be included. If any “exclude” condition is met then the data point is not included.
Each condition consists of two value specifications separated by one of “<”, “⇐”, “>”, “>=”, “==”, “!=”, or “&”, “!&”. The condition evaluates to true if the first value is less than, less than or equal to, greater than, greater than or equal to, exactly equal, not equal, a bitwise AND evaluated to non-zero, or evaluates to zero.
The special values “true” and “false” are also supported causing the require or exclude to always evaluate to the given boolean value.
Function
Set the function used to fit the data. One of:
- GammaOne - (1-RH/100)^(-gamma)
- GammaTwo - a*(1-RH/100)^(-gamma)
- Linear - a + b * RH (testing only, not representative of real data)
- LinearGamma - a + b * (1-RH/100)
- Quinn - a*( (100-RHwet)/(100-RHdry) )^(-gamma)
Defaults to “GammaTwo”.
This defines the output parameters generated. All fits are to the ratio of wet over dry scatterings.
Method
Set the fitting routine used. One of:
- LevenbergMarquardt - Levenberg-Marquardt_algorithm
This defines the goodness of fit generated.
Priority
The internal priority that the fit has. This is used primarily for generating the ordering of variables. Fits with a smaller priority generate lower numbered instances of the variables. Fits with no priority set always sort to the end and fits with equal (or both absent) priorities are sorted alphanumerically.
Common Parameters
These parameters can be set in either the profile block or on a per-fit basis in a fit block. If they are set in the profile block, they override the default for all fits.
OutputInstrument
The instrument code used in the output records. Defaults to “XH”.
OutputRecord
The output record generated. Defaults to the “a” record for the instrument.
RequiredPoints
The minimum number of valid points required before a fit is attempted. Defaults to 14.
MinimumHighRH
The minimum value for the highest RH achieved before a fit is attempted. Defaults to 60. That is, a fit will not be done if the highest RH reached is less than 60.
MaximumLowRH
The maximum value for the lowest RH achieved before a fit is attempted. Defaults to 64. That is, a fit will not be done if the lowest RH reached is greater than 64.
MinimumSpanRH
The minimum difference between the highest and lowest RHs achieved before a fit is attempted. Defaults to 30. That is, a fit will not be done if the difference between the highest and lowest RH reached is less than 30.
MinimumDryBsG
The minimum average green scattering before a fit is attempted. Defaults to 5. That is, if the average green scattering for the entire fitting period is less than 5, a fit will not be done.
ExcludeWetRH
Exclude all points for which the wet RH is less than this value. Defaults to 40. That is, points with a wet RH of less than 40 will not be considered in the fit.
ScanLength
Set the length of the scan in seconds. Defaults to 3600. That is, a fit will be done once per hour on the hour.
Value Specifications
A value specification normally consists of a variable and an optional calibration. The components are separated by “:”, “;”, or “,” (be sure to quote comma containing elements, however). The variable comes first and must be present in one the input records (see above to request additional records). The calibration starts at the offset, then the first order term and so on. For example:
U_V11:0.5:1
Is the sensor “U_V11” offset by +0.5% RH. The special variables “T_wet”, “T_dry”, “U_wet”, “U_dry”, “scan” are available to represent the temperature, humidities, and seconds into the scan.
The specification can also be a simple decimal constant (e.x. “1.5”) or specification of the form “time” or “time:modulus” where the time is the number of seconds after the start of the scan, optionally modulo the given number.
Example
Aerosol Dry Instrument,S11 END Wet Instrument,S12 Records,X1a,X2a Td,T_V13,U_V13 END Fits Global Priority,0 Description,"all data" END Low Priority,1 Description,"RH < 65 percent" Require,U_wet<65 RequiredPoints,7 MinimumSpanRH,15 MinimumHighRH,55 END High Priority,2 Description,"RH > 65 percent" Require,U_wet>65 RequiredPoints,7 MinimumSpanRH,15 MaximumLowRH,70 END End END
This configuration defines the fit as using “S11” for the dry instrument with all defaults. “S12” for the wet instrument but recalculating the internal RH using an external sensor for the dewpoint and the internal temperature sensor. It defines three fits, one for all data using all defaults, one for low (<65%) and one for high (>65%) RH. These fits override the default filtering condition to be in line with the data they contain. The RH used is the one calculated from the dewpoint.