hMon DMT Monitor

WARNING: hMon is currently under development, as is this documentation.

Overview | Data products | Command line arguments | Configuration | Environment variables | Source code | Building and Testing | Acknowledgements | References

Overview

hMon is a near real-time generator of time-domain calibrated data and baseline segment lists for rapid online analysis.

hMon is a Data Monitor Tool (DMT) monitor that can read in data from either the online shared memory partition or from frame files on disk.

Data products

In addition to time-domain calibrated data, hMon produces a number of other data products, which are made available in a variety of ways.

Time-domain calibrated data

Time-domain calibrated data is the primary data product of hMon, and is made available in three different ways.

frame files on disk
Calibrated data is written to short duration frame files with a pre-determined name in order to simplify data discovery.
shared memory partition
Calirbated data is also made available via a shared memory partition for use by other monitors. This is not yet implemented.
network data server
Calibrated data is made available via the network data server (NDS2) protocol in order to stream data to remote locations. This is not yet implemented.

Segment Lists

While computing time-domain calibrated data, hMon keeps track of which data has been read, which data has been processed, and which data should be excluded due to start and stop filter transients. While hMon processes all available daeta, it also identifies peridos of bad caibration or missing data. hMon also monitors detector state to determine in lock segments, science mode segments, and injection segments.

list namedescription
readread by hmon
missingperiods of missing data
transientstart and stop transients to exclude
badperiods of bad calibration
lockin lock segments
sciencescience mode segments
injectioninjection segments

These segment lists are combined to form two primary baseline segment lists for rapid online analysis.

list namedescription
calibratedvalid calibrated data to analyze (all processed in-lock data excluding transients)
vetoperiods to exclude a posteriori (periods of non-science mode, injection mode, saturation, or bad calibration, padded by filter transients)

These segment lists can be further supplemented by more extensive data quality studies, once those lists become available.

Segments are aligned to second boundaries and follow the convention [startTime, stopTime), in that they are inclusive of the start time and exclusive of the stop time.

Segments are made available in two ways:

database publication
Segments are published directly to the database used by LSCsegFind. New segments are published once per frame stride. Coalescence of adjaced segments is left as a database maintenance task.
http publication
Segments are written to a file, which can be made available locally via NSF and globally via http. Segments are coalesced, but only segments from the last segmentPersistence days are made available.

Link to LSCsegFind documentation

Format of text segment lists

Calibrated spectra

Sensitivity

Trends

Detector state

Status reports

Alarms

Command line arguments

hMon currently takes the following arguments.

  hMon \
    -configuration configurationFile \
    [-verbose verboseLevel] \
    [-partition partitionName] \
    [-infile frameFiles] \
    [-inlist listFile] \
    [-debug debugLevel] \
    [-maxframes maximumValue] \

The configurationFile argument is required. See below for configuration file documention.

The verboseLevel argument controls the level of detail sent to the standard output by hMon. By default it is set of level 1.

The remaining arguments are to the underlying DatEnv class. The input data is read from frame files if the infile or inlist options are specified, and from the online system if the correct partiion is specified. If no input option is specified, the frames are read from the partition specified by the LIGOSMPART environment variable or, if this is not defined, from "LIGO_Online".

Configuration

The configuration file should specify the following parameters. Example values are given for H2.

  # hMon monitor configuration file for H2

  # detector name
  detectorName           H2

  # input sample frequency
  sampleRate             16384
  upSampleFactor         1

  # fundamental time step
  unitStride             1

  # output frame file properties
  frameStride            10
  frameType              <DETECTOR>_HMON
  frameDirectory         output/<DETECTOR>/frames
  frameEpochDuration     10000
  frameChannelName       <DETECTOR>:LSC-STRAIN
  frameSampleRate        16384
  frameCompression       1
  framePrecision         single

  # segment list properties
  detectorStatesFile     input/<DETECTOR>/config/states.txt
  segmentDirectory       output/<DETECTOR>/segments
  segmentListDuration    259200
  segmentVersion         1

  # input channel names
  sensingChannelName     <DETECTOR>:LSC-AS_Q
  errorChannelName       <DETECTOR>:LSC-DARM_ERR
  controlChannelName     <DETECTOR>:LSC-DARM_CTRL
  excitationChannelName  <DETECTOR>:LSC-DARM_CTRL_EXC_DAQ

  # filter file names
  sensingFilterFile      input/<DETECTOR>/filters/sensing.txt
  servoFilterFile        input/<DETECTOR>/filters/servo.txt
  actuationFilterFile    input/<DETECTOR>/filters/actuation.txt
  lowPassFilterFile      input/<DETECTOR>/filters/lowpass.txt
  highPassFilterFile     input/<DETECTOR>/filters/highpass.txt

  # first calibration line
  excitationFrequency    100
  excitationAmplitude    3

  # second calibration line
  excitationFrequency    200
  excitationAmplitude    2

  # third calibration line
  excitationFrequency    500
  excitationAmplitude    1

  # server properties
  serverDuration         28800

  # spectrum estimation
  spectrumResolution     1
  spectrumDecayTime      10

  # threshold for latency warning
  latencyThreshold       0

The following sections describe these parameters in more detail.

Detector name

The parameter detectorName is a required parameter. It must be one of the two character detector identifies H1, H2, or L1.

detectorName           H2

Any occurence of the string <DETECTOR> in other parameters is replaced by this detector name.

Timing parameters

unitStride             1
frameStride            10

Channel name parameters

sensingChannelName     <DETECTOR>:LSC-AS_Q
errorChannelName       <DETECTOR>:LSC-DARM_ERR
controlChannelName     <DETECTOR>:LSC-DARM_CTRL
excitationChannelName  <DETECTOR>:LSC-DARM_CTRL_EXC_DAQ

Filter parameters

sensingFilterFile      input/filters/sensing.txt
servoFilterFile        input/filters/servo.txt
actuationFilterFile    input/filters/actuation.txt
lowPassFilterFile      input/filters/lowpass.txt
highPassFilterFile     input/filters/highpass.txt

Frame file parameters

There are a number of parameters that control the writing of frame files containing the time-domain calibrated data.

frameStride            10
frameType              <DETECTOR>_HMON
frameDirectory         output/frames/<DETECTOR>
frameChannelName       <DETECTOR>:LSC-STRAIN
frameSampleRate        16384
frameCompression       10
framePrecision         single
frameEpochDuration     10000

The resulting frames data are written to files with the following naming convention.

frameDirectory/siteName-detectorName_frameType-epochTime/siteName-detectorName_frameType-startTime-frameStride.gwf

The site is determined by taking the first character of the detector name.

The epochTime is determined by integer division of the startTime by the frameEpochDuration.

In order to simplify data discovery, the startTime of frame files are always aligned such that they are an integer multiple of the frameStride. In addition, all data is processed, regardless of detector state, and all frame files are of duration frameStride. Any missing data are replaced with zeros.

A typical frame file name, given the example configuration above, would be similar to the following.

output/frames/H-H2_HMON-87654/H-H2_HMON-876543210-10.gwf

It is an error if the specified frameDirectory does not exist.

The data in the frame file consists of the channel frameChannelName, which contains the output time-domain calibrated data. This data has a sample frequency of frameSampleRate Hz, and is written single or double precision as specified by the framePrecision parameter. The data is also compressed, depending on the value of the frameCompression parameter, as listed in the following table.

valuedescription
0uncompressed
1gzip
3gzip differential
5zero supress 2
8zero supress 4
10gzip zero supress

See email from John regarding compression modes.

Segment parameters

detectorStatesFile     input/<DETECTOR>/config/states.txt
segmentDirectory       output/<DETECTOR>/segments
segmentListDuration    259200
segmentVersion         1

The detectorStatesFile parameter points to the Operational State Condition (OSC) configuration file that specifies the definitions of the LOCK, SCIENCE, INJECTION, and SATURATION conditions used by hMon.

The following is an example detector states configuration file for H2.

# hMon operational state condition configuration file for H2
# $Id: hMon.html,v 1.7 2008/09/05 16:39:10 shourov Exp $

LOCK       bitand   "H2:IFO-SV_STATE_VECTOR" mask=0x1C fraction=1.0
STRICT     bitand   "H2:IFO-SV_STATE_VECTOR" mask=0x1F fraction=1.0
SCIENCE    bitand   "H2:IFO-SV_STATE_VECTOR" mask=0x17 fraction=1.0
OVERFLOW   meanrise "H2:LSC-MASTER_OVERFLOW" threshold=0 nstrides=2
INJECTION  boolean  "SCIENCE & !STRICT"
SATURATION boolean  "OVERFLOW & LOCK"

Only recent segments are reported by hMon. The segmentListDuration parameter specifies the maximum duration of segment lists written by hMon. If this value is set of 0, all segments are reported.

The segmentVersion is the version number applied to segments inserted into the database.

Excitation line parameters

Any number of excitation lines may be specified.

excitationFrequency    100
excitationAmplitude    3

Alarm parameters

latencyThreshold       0

Spectrum and sensitivity parameters

spectrumResolution     1
spectrumDecayTime      10

Environment

There are a number of environment variables that affect DMT Monitors and need to be set prior to running.

The following environment variables affect hMon.

DMTNAMESERVER
DMTRENDOUT
DMTHTMLOUT

Source code

The hMon source code is maintained in the GDS CVS repository.

Building and testing

To build the code, you will need to build both Root and href="http://www.ligo-wa.caltech.edu/gds/">GDS.

Instructions for building ROOT and GDS.

Prebuilt DMT binary packages for offline use are provided as provided as part of the DMT OffLine (DOL) project.

Instructions for running enclosed test script

Acknowledgements

The developers would like to acknowledge the following people who have contributed or influenced this code.

The algorithms for time-domain calibration of data were taken from the LAL based h(t) production approach of Xavier Siemens et al. [1, 2, 3], which in turn was based on earlier work by Martin Hewitson et al. for the GEO 600 detector[4, 5].

Ideas and inspiration have also been taken from Ramon Armen's StrainWatch[6] DMT monitor and Patrick Sutton, Kevin Schlaufman, and Aaron Rogan's EasyCalibrate[7] DMT class.

The approach to estimating sensitivity has been taken from Patrick Sutton and Kevin Schlaufman's SenseMonitor DMT monitor[8].

The idea for DMT generation of data quality flags for periods of bad calibration came from the Detector Characerization working group chairs Keith Riles and Gaby Gonzalez [9].

Much support has been provided by John Zweizig as primary developer and maintainer for the DMT environment, including the development of a new class for frequency domain application of FIRFilters, and integration of hMon with the NDS2 protocal for streaming data to remote locations.

References

1. X. Siemens, B. Allen, J. Creighton, M. Hewitson and M. Landry, "Making h(t) for LIGO," Class. Quant. Grav. 21, S1723 (2004) [arXiv:gr-qc/0405070].

2. LALApps h(t) code documentation (see directories → lalapps → calibration).

3. Time-domain calibration how-to guide.

4. M. Hewitson et al, 2003 Rev. Sci. Instrum. 74 4184 .

5. M. Hewitson et al, 2003 Class. Quantum Grav. 20 1 .

6. StrainWatch DMT Monitor (description, code).

7. EasyCalibrate DMT Class (code)

8. SenseMonitor DMT Monitor (description, code).

9. Plans for migrating S5 offline DQ flags to S6 online DMT monitors or cron scripts.

10. LIGO Amplitude Calibration Homepage.


$Id: hMon.html,v 1.7 2008/09/05 16:39:10 shourov Exp $