Package glue :: Package ligolw :: Module ilwd
[hide private]
[frames] | no frames]

Module ilwd

source code

The ilwd:char type is used to store ID strings for objects within LIGO Light-Weight XML files. This module and its associated C extention module _ilwd provide a class for memory-efficient storage of ilwd:char strings.

LIGO Light Weight XML "ilwd:char" IDs are strings of the form "table:column:integer", for example "process:process_id:10". Large complex documents can have many millions of these strings, and their storage represents a significant RAM burden. However, while there can be millions of ID strings in a document there might be only a small number (e.g., 10 or fewer) unique ID prefixes in a document (the table name and column name part). The amount of RAM required to load a document can be significantly reduced if the small number of unique string prefixes are stored separately and reused. This module provides the machinery used to do this.

The ilwdchar class in this module converts a string or unicode object containing an ilwd:char ID into a more memory efficient representation.

Example:

>>> x = ilwdchar("process:process_id:10")
>>> print(x)
process:process_id:10

Like strings, the object resulting from this is immutable. It provides two read-only attributes, "table_name" and "column_name", that can be used to access the table and column parts of the original ID string. The integer suffix can be retrieved by converting the object to an integer.

Example:

>>> x.table_name
u'process'
>>> int(x)
10

The object also provides the read-only attribute "index_offset", giving the length of the string preceding the interger suffix.

Example:

>>> x.index_offset
19

The objects support some arithmetic operations.

Example:

>>> y = x + 5
>>> str(y)
'process:process_id:15'
>>> int(y - x)
5

The objects are pickle-able.

Example:

>>> import pickle
>>> x == pickle.loads(pickle.dumps(x))
True

To simplify interaction with documents that do not contain fully-populated columns, None is allowed as an input value and is not converted.

Example:

>>> print(ilwdchar(None))
None

Implementation details

Memory is reduced by storing the table_name, column_name, and index_offset values as class attributes, so only one copy is present in memory and is shared across all instances of the class. This means that each unique table_name and column_name pair requires its own class. These classes are created on the fly as new IDs are processed, and get added to this module's name space. They are all subclasses of _ilwd.ilwdchar, which implements the low-level machinery. After a new class is created it can be accessed as a symbol in this module, but each of those symbols does not exist until at least one corresponding ID string has been processed.

Example:

>>> import ilwd
>>> "foo_bar_class" in ilwd.__dict__
False
>>> x = ilwd.ilwdchar("foo:bar:0")
>>> type(x)
<class 'glue.ligolw.ilwd.foo_bar_class'>
>>> "foo_bar_class" in ilwd.__dict__
True
>>> print(ilwd.foo_bar_class(10))
foo:bar:10

The ilwdchar class itself is never instantiated, its .__new__() method parses the ID string parameter and creates an instance of the appropriate subclass of _ilwd.ilwdchar, creating a new subclass before doing so if neccessary.


Version: git id 8cbd1b7187ce3ed9a825d6ed11cc432f3cfde9a5

Date: 2017-12-05 15:29:36 +0000

Author: Kipp Cannon <kipp.cannon@ligo.org>

Classes [hide private]
  coinc_definer_coinc_def_id_class
  coinc_event_coinc_event_id_class
  dq_list_dq_list_id_class
  dq_list_dq_list_row_id_class
  experiment_experiment_id_class
  experiment_summary_experiment_summ_id_class
  filter_filter_id_class
  gds_trigger_event_id_class
  ilwdchar
Metaclass wrapper of glue.ligolw._ilwd.ilwdchar class.
  lfn_lfn_id_class
  ligolw_mon_event_id_class
  multi_inspiral_event_id_class
  process_process_id_class
  search_summvars_search_summvar_id_class
  segment_definer_segment_def_id_class
  segment_segment_id_class
  segment_summary_segment_sum_id_class
  sim_burst_simulation_id_class
  sim_inspiral_simulation_id_class
  sim_inst_params_simulation_id_class
  sim_ringdown_simulation_id_class
  sngl_burst_event_id_class
  sngl_inspiral_event_id_class
  sngl_ringdown_event_id_class
  summ_mime_summ_mime_id_class
  summ_value_summ_value_id_class
  time_slide_time_slide_id_class
Functions [hide private]
 
get_ilwdchar_class(tbl_name, col_name, namespace={'__author__': 'Kipp Cannon <kipp.cannon@ligo.org>', '__builti...)
Searches this module's namespace for a subclass of _ilwd.ilwdchar whose table_name and column_name attributes match those provided.
source code
Variables [hide private]
  __package__ = 'glue.ligolw'
Function Details [hide private]

get_ilwdchar_class(tbl_name, col_name, namespace={'__author__': 'Kipp Cannon <kipp.cannon@ligo.org>', '__builti...)

source code 

Searches this module's namespace for a subclass of _ilwd.ilwdchar whose table_name and column_name attributes match those provided. If a matching subclass is found it is returned; otherwise a new class is defined, added to this module's namespace, and returned.

Example:

>>> process_id = get_ilwdchar_class("process", "process_id")
>>> x = process_id(10)
>>> str(type(x))
"<class 'glue.ligolw.ilwd.process_process_id_class'>"
>>> str(x)
'process:process_id:10'

Retrieving and storing the class provides a convenient mechanism for quickly constructing new ID objects.

Example:

>>> for i in range(10):
...     print str(process_id(i))
...
process:process_id:0
process:process_id:1
process:process_id:2
process:process_id:3
process:process_id:4
process:process_id:5
process:process_id:6
process:process_id:7
process:process_id:8
process:process_id:9