DST

• I3DSTModule
• I3DSTExtractor
• DST Scripts
• icecube.dst package
• dst C++ API Reference
• Project dst
• Release Notes

Introduction

The object of the DST project is to collect information about all the events recorded by the IceCube array. The purpose of having such high statistics is multiple; It can be used for high statistics diagnostics of the detector performance, and also for specific physics analyses that may require us to collect information about all the triggered events. The DST (Data Storage & Transfer) on-line filtering stream runs at the South Pole as part of PnF (process and filter), where all triggered events are reconstructed using fast algorithms and then only a limited number of observables is written to disk in a compact format. DST records the fast reconstruction results implemented in the Pole Filtering scheme and collects muon track reconstruction results for each event, along with other observables like the number of hit DOMs, the number of pulses/hits (reconstructed by FeatureExtractor), the event time, and some trigger information.

This DST event information is collected and written to a I3DST object derived from I3FrameObject. The filtering tools for this specific stream are fast and the resulting data is sufficiently compact such that we are able collect information about all events that trigger IceCube while complying with restrictions of CPU power and data transmission bandwidth.

IceTray Modules

The dst project contains a Module (I3DSTModule) that can write the DST filtering information directly in the frame, thus avoiding the introduction of an extra file. In this case the DST event information is written in each Physics frame as I3DST, an I3FrameObject. The I3DSTExtractor module can take the DST information from the frame and write DST information according to the structure described above in a ROOT or HDF5 file. It can also partially reconstruct the I3 dataclasses from which I3DSTModule extracted the DST information.

The size of the DST filtering information depends on the writing mode and the compression level. If writing to dst format binary file, obviously we have 24 Bytes per event size without compression and 6-9 Bytes / event if using zlib-compression (more efficient than gzip). Using simulation data we get about 6 B. If writing to the Frame the I3DST is 44 Bytes per event. With gzip compression of the i3 files the I3DST are about 10 Bytes.

I3DST format

The DST generated by the Online Filter is stored as an I3FrameObject in .i3 files. Since each event is processed asynchronously on the client nodes there is no way to enforce the placement of an I3DSTHeader object on the first frame. Therefore, the I3DST will contain a 32bit integer containing the absolute time instead of the delta_t used in the .dst format. I3DST in I3Frame

I3DST Event

description	variable	size (B)	type
Trigger Tag		2	uint16_t
Fast Reco #1	reco1	2	DSTReco
Fast Reco #2	reco2	2	DSTReco
IC Hits	ic_nhits0	1	uint8_t
	ic_nhits1	2	uint16_t
clean nDOMs	ic_ndom	2	uint16_t
nStrings	ic_nstring	1	uint8_t
ndirC	ic_ndir	1	uint8_t
ldirC	ic_ldir (10 m)	1	uint8_t
rlog	ic_rlog (0.1)	1	uint8_t
logE	ic_logE (1)	1	uint8_t
Time 100 \({\mu}s\)	time	4	uint32_t
Hits COG	cog (in 10 m)	3	DSTPosition(uint8_t,uint8_t,uint8_t)
reco label	bit array of recos	1	uint8_t
Total	24

DSTReco

description	variable	size (B)	type
Sky coord	coord	2	uint16_t
Total	2

Track reconstruction algorithms provide track directions (theta, phi), which are float numbers (4 + 4 B = 8 B per event). Instead of saving the track direction directly with the pairs of angles, we map the entire sky with a grid of equal solid-angle bins, similar to the one used in point-source searches) and we assign each (theta, phi) pair a unique point coordinate ID (point_coord). A description of how the sky grid is generated and how pairs of angles are mapped to an integer ID is given in Appendix B. The reconstruction results to be included are, most probably, LineFit and LLHReco if InIce ONLY trigger is present. The number of hit DOMs (nch) and of hits (nhits) are a direct measurement of the energy deposited by the event in the detector. For nhits we use a pair of (1B, 2B) integers, since a 2 B word is too small and a 4 B integer is too big. These 3 B cover the possible range. The 1 B number contains the most significant portion of the number and the 2 B number the remaining. The event time is saved using the driving time T_d in the Header and the time difference Dt between adjacent events in each event. In this way the time of the first event if t_1 = T and the time of the ith event is \({t_i =t_{i-1} + \Delta t}\).

The driving time from the header is the time from January 1st of the year. This is extracted from the first event in the file and each subsequent event time is calculated with respect to this driving time. The time difference between adjacent events is a 1 B number and it can measure up to 255 1/100 of a second (i.e. 2,550 msec). Given a trigger rate of ~400 Hz this means that the average time between events is ~2.5 msec . The probability that a 400 Hz event sequence give an event separation of 2,550 msec is extremely small, therefore the size of the Dt number (1 Byte) is enough. Nevertheless there might be sporadic cases where the run is still going while no event is recorded for technical reasons. In this case we provide a safeguard and in the eventuality that two subsequent events are be separated by more than 2,550 msec within the same file, a new Header Binary Segment is introduced which resets a new driving time. The reco label field stores a bit array which indicates which reconstructions are stored in the frame. This assumes that you know what reconstructions were configured at filter time since it only gives you the index of the reconstruction. For example, if you configured I3DSTModule with the following:

tray.AddModule("I3DSTModule","dst")(
  ("RecoList", ["llh","LineFit", "ToI"])
)

A mapping of indices to actual reconstruction names is stored in the I3DSTHeader.

Appendix

A: Trigger tag Information

We extract the relevant information from the IceCube Trigger Hierarchy structure and write it in a 2 B unsigned integer in a compact form. We want to keep track of the following Trigger info:

• Source ID:

  • IN_ICE (IC)

  • ICE_TOP (IT)

  • SPASE (SPASE)

• Type ID:

  • SIMPLE_MULTIPLICITY (SM)

  • MIN_BIAS (MB)

  • STRING (STR)

  • VOLUME (VOL)

  • SPHERE (SPH)

  • FRAGMENT_MULTIPLICITY (FM)

  • UNKNOWN (UK)

We also know that not all the trigger sources are associated to all the trigger types. We can only have the following source-type ID associations :

Trigger Table
Source ID	Type ID’s
IC	SM	MB	STR	VOL	SPH
IT	SM	MB
bit	0	1	2	3	4	5	6

B: Sky grid for directional information (HEALPix)

The reconstructed track direction could be saved in the DST event as a pair of floats (theta, phi). A way to save it in a more compact manner is to map the (theta, phi) pair into a 2 Byte signed integer (int16_t) ID. To do that we generate a grid in the sky, and we assign the ID to each bin. Instead of generating our own, we are now relying on HEALPix (https://healpix.jpl.nasa.gov) to do this. HEALPix partitions the surface of the celestial sphere into pixels such that each pixel covers the same surface area as every other pixel. Once the sky grid is constructed for the entire celestial sphere, each reconstructed (theta, phi) pair is mapped to an integer which uniquely identify each bin in the grid.