The Data Distribution sub-system receives real-time interferometer data from the CDS Network Data Server (NDS) and distributes it by way of shared memory buffers to any Data Monitor Applications that may be running.
The Data Distributor is comprised of the following components:
RELBUF Modifier: A full buffer is released immediately to the free list if there are no pending requests from a consumer. SCAVAGE Modifier: A buffer in the full list that is not in use and is not reserved for any consumer may be allocated to a producer if no buffers are available on the free list. In this case, the oldest available full buffer is used. RQSYNC Modifier: A buffer is allocated to a producer only if there is a pending request for a buffer by a consumer.
The consumer interface allows the consumer to reserve any number of buffers. By default no buffers are reserved for the consumer. When this is the case, each time a consumer requests a buffer the consumer interface takes the oldest buffer in the partition's full-list. If there are no buffers in the full-list, the consumer request fails or the consumer waits for the next available buffer. When the buffer is released it is marked as having been seen by the consumer to prevent it from being reread with the next request.
If the consumer requests that one or more buffers be reserved for it, the (shared) reservation of the buffer is performed at the time the buffer is distributed by the producer. The buffer will be reserved for all consumers that have selected the buffer by the trigger mechanism and have not filled their specified quota of reserved buffers. The reservation is canceled when the consumer releases the buffer.
|
|
Producer Overhead |
|
Consumer Overhead |
|
Process Swap |
|
Write data rate |
|
Read data rate |
|
The interface to FrameL is by way of a
redefined FrIO structure and
the associated interface functions FrIOClose(),
FrIOOpen(), FrIORead() and
FrIOWrite(). These are defined
in ONLFrIO.h and ONLFriIO.cc.
Note that ONLFrIO is implemented
as a C++ package with a C interface in order to communicate with both FrameL
and LSMP.
The data pusher programs receive data from various data sources and copy them to a shared memory buffer. One or the other program must be running at all times in order to provide data to the monitor programs. Several data pushers may be run in parallel on different partitions to provide multiple data sets to different monitor groups. Care should be taken not to start more than one Dpush process on a single partition. as this might cause a monitor to receive duplicate copied of a single data frame. The three Dpush programs read data from different sources, but otherwise function in much the same manner. DpushF reads frames from the CDS Network Data Server (NDS), DpushM receive frames broadcast by the Frame Broadcaster, and DpushRT replays frames stored on spinnig media in real time. On startup, Dpush will create a shared memory partition with the specified name, length and number of buffers if the named partition does not already exist. It then copies data from the specified source to the shared memory partition. It also keeps statistics on the number of missing Frames and the average rate of arrival of the frames and prints them to a log file every 20 minutes. The syntax for the Dpush command is the following:
Dpush [NDS <node>]
[infile <file>] [partition <pname>] [lbuf <nby>] [nbuf <n>]
\
[log <logfile>] [time <ticks>] [debug]
The arguments to Dpush
are:
<node>
Node name or IP address of the network
data server (DpushF, DpushM).
<file>
Input file name or wild-card expresion.
(DpushRT only)
<pname>
Name of shared memory partition to
receive the data.
<nby>
Minimum buffer length for the partition.
<n>
Minimum number of buffers in the partition.
<logfile>
Name of file to receive error and statistics
messages.
<ticks>
Number of seconds between statistics
summaries.
Entering an invalid argument causes a help message to be printed out. The Dpush executables reside in the $DMTHOME/bin directory.
When running under DMTrestart, DpushF can usually be upgraded without affecting any running monitors by replacing the DpushF executable and killing the process. It will be restarted with the loss of at most 1-2 frames, without otherwise affecting the client processes.Make sure that a copy of DpushF is not already running. Check that the NDS server is alive and responds to a version number request. If not, wait for it to start, testing in 20s intervals. Generate a log file name of the form $HOME/logs/DpushF-yyyy.mm.dd-hh.mm. Start DpushF with all the appropriate arguments. Wait for the DpushF to fail/terminate.
A set of utility programs is has been written for maintaining the Shared memory partitions. These are listed below:
smcreate <partition> [-nbuf <n>] [-lbuf <length>]
The parameters to smcreate are:
<partition>
Name of the partition to be created.
<n>
Number of buffers in partition.
<length>
Length of buffers.
smdump [<partition>]
The parameter to smdump is:
<partition>
Name of the partition to be dumped. If
<partition>
is not specified, the
partition name will be taken from the
LIGOSMPART
environment variable.
smraw <partition> [-nwmax <nw>]
The parameters to smraw are:
<partition>
Name of the partition to be dumped. If
<partition> is
not specified, the
partition name will be taken from the
LIGOSMPART
environment variable.
<nw>
Number of (32-bit) words to be dumped
per record, by default <nw>
is 64.
smrepair <partition>
The parameter to smrepair is:
<partition>
Name of the partition to be repaired.
If <partition> is
not specified, the
partition name will be taken from the
LIGOSMPART
environment variable.
buffer_tot Total number of buffers read into partitionThe command syntax is:
freebuf Number of buffers in the free list
fullbuf Number of buffers in the full list.
last_id Data ID of most recent buffer.
lbuf Maximum buffer length.
maxcons Maximum number of consumers.
name Partition name
nbuf Number of buffers allocates in the partition.
ncons Number of active consumers.
pflags Partition flag bits.
semglobal Global semaphore ID.
shmid Shared memory partition ID.
use_count Number of processes using the shared memory partition.
usedbuf Number of in-use buffers.
version LSMP version that created the partition.
smstat <param1> [<param2> ...] <partition>Where <paramx> is a parameter name from the list, and <partition> is the name of the shared memory partition to be interrogated.