icecube.gulliver_modules.fortytwo module¶
Test suite for Gulliver
Implementation of an IceTray module to test reproducibility of processing results.
Frame object values from current processing are compared with values obtained with the same script in the past and/or on some other platform.
The primary (and currently only) use case is to include this module in the resource/test scripts that are run by the build bots. Failure to reproduce results results in un-green checkboxes.
If this ever turns out to become more popular then it should move to some project in offline-software. Then it should also be split up, each class in a separate file.
- class icecube.gulliver_modules.fortytwo.DummyChecker¶
Bases:
I3FrameChecker
This checker just serves as an example of a very simple implementation of an
I3FrameChecker
. It computes only one test value.- compare_test_values(values, reference_values)¶
This method must be implemented by a subclasss. The values and reference_values are both lists of test values as computed by the
get_test_values
method.If some test values are NOT compatible with each other then a string should be returned with an informative description of the problem(s).
If the current test values and the reference ones are are all compatible with each other then an empty string (or
None
, orFalse
) should be returned.
- get_test_values(index, frame)¶
This method must be implemented by a subclasss. For every frame the same number of test values are computed, which should be returned in the form of a list.
- class icecube.gulliver_modules.fortytwo.I3FortyTwo(context)¶
Bases:
I3ConditionalModule
Check processing results
A module to check that processing results (e.g. track fits, multiplicities) are the same on different platforms and do not change over time due to changes in the processing software. Reference results for any number of processing/testing scripts can be stored in a single picklefile. This module only cares about the final answers of other module(s) and that they have the expected value(s). Hence the name.
An I3FortyTwo module operates in two different modes:
- ‘reference’: test results of the current run of the script are
stored in a pickle file. This pickle file can be stored in SVN and then be used by for instance the build bots.
‘check’: test results from the latest reference run are read in from the pickle file. They are also computed for the current run, and tested for equality (or equivalence). If any (too large) deviations then the Finish() method will throw an exception in order to trigger the build bots to un-green the appropriate checbox on builds.icecube.wisc.edu.
- Check(frame)¶
- Configure((I3ConditionalModule)arg1) None : ¶
- C++ signature :
void Configure(PythonModule<I3ConditionalModule> {lvalue})
- Finish((I3ConditionalModule)arg1) None : ¶
- C++ signature :
void Finish(PythonModule<I3ConditionalModule> {lvalue})
- Physics((I3ConditionalModule)arg1, (I3Frame)arg2) None : ¶
- C++ signature :
void Physics(PythonModule<I3ConditionalModule> {lvalue},boost::shared_ptr<I3Frame>)
- ReadPickle()¶
- Reference(frame)¶
- class icecube.gulliver_modules.fortytwo.I3FrameChecker¶
Bases:
object
This class serves to define the interface for checks on specific (types of) objects within a frame, to be used by an
I3FortyTwo
module.- compare(index, frame)¶
This method gets run in check mode. The checker object is then instantiated by reading it from a pickle file, so the reference_list should be filled with the test values from a previous run. We compute the test values now again in the current run, and check that they are compatible with the values stored in reference_list.
- compare_test_values(values, reference_values)¶
This method must be implemented by a subclasss. The values and reference_values are both lists of test values as computed by the
get_test_values
method.If some test values are NOT compatible with each other then a string should be returned with an informative description of the problem(s).
If the current test values and the reference ones are are all compatible with each other then an empty string (or
None
, orFalse
) should be returned.
- finish()¶
Optional, write diagnostic stuff at the end.
- get_test_values(index, frame)¶
This method must be implemented by a subclasss. For every frame the same number of test values are computed, which should be returned in the form of a list.
- num_reference_events()¶
Return the number of reference events. All of them should be checked.
- reference(index, frame)¶
This method gets called by the
I3FortyTwo
module in reference mode. The checker object is then installed in the processing script and passed to the module via the configurable parameters. We compute the test values and store them within this checker object in the reference_list.
- report_missing(key)¶
- class icecube.gulliver_modules.fortytwo.I3LogLikelihoodFitParamsChecker(params=[])¶
Bases:
I3FrameChecker
Checks the diagnostics object of gulliver fits.
Diagnostics:
likelihood value
reduced likelihood value
number of degree of freedom
number of minimizer steps
- atols = (0.05, 0.05, 0.0, 100.0)¶
- compare_test_values(values, reference_values)¶
This method must be implemented by a subclasss. The values and reference_values are both lists of test values as computed by the
get_test_values
method.If some test values are NOT compatible with each other then a string should be returned with an informative description of the problem(s).
If the current test values and the reference ones are are all compatible with each other then an empty string (or
None
, orFalse
) should be returned.
- get_test_values(index, frame)¶
This method must be implemented by a subclasss. For every frame the same number of test values are computed, which should be returned in the form of a list.
- problems(values, reference_values, mask)¶
- rtols = (0.01, 0.01, 0.0, 0.0)¶
- class icecube.gulliver_modules.fortytwo.I3ParticleChecker(particles=[])¶
Bases:
I3FrameChecker
Checks fit status and direction of I3Particle.
Needs a parameter to tweak the tolerance at which floating point numbers should be compared (with
numpy.isclose
).- atols = (0.0, 0.0, 0.0, 0.02, 0.02, 0.1, 0.1, 0.1, 0.1, 0.1, 0.01)¶
- compare_test_values(values, reference_values)¶
This method must be implemented by a subclasss. The values and reference_values are both lists of test values as computed by the
get_test_values
method.If some test values are NOT compatible with each other then a string should be returned with an informative description of the problem(s).
If the current test values and the reference ones are are all compatible with each other then an empty string (or
None
, orFalse
) should be returned.
- get_test_values(index, frame)¶
This method must be implemented by a subclasss. For every frame the same number of test values are computed, which should be returned in the form of a list.
- problems(values, reference_values, mask)¶
- rtols = (0.0, 0.0, 0.0, 0.001, 0.001, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01)¶
- icecube.gulliver_modules.fortytwo.MultiplicityCutModule(frame, pulses, nchmin=0, nchmax=10000, nstrmin=0, nstrmax=10000)¶
- class icecube.gulliver_modules.fortytwo.Pcount(context)¶
Bases:
I3ConditionalModule
Progress meter
Silly for default test, but useful if you use the cmdline options to e.g. run this on a larger input file. There is an I3EventCounter in phys-services, but that is clunky. It requires an XML files service.
- Configure((I3ConditionalModule)arg1) None : ¶
- C++ signature :
void Configure(PythonModule<I3ConditionalModule> {lvalue})
- DAQ((I3ConditionalModule)arg1, (I3Frame)arg2) None : ¶
- C++ signature :
void DAQ(PythonModule<I3ConditionalModule> {lvalue},boost::shared_ptr<I3Frame>)
- Physics((I3ConditionalModule)arg1, (I3Frame)arg2) None : ¶
- C++ signature :
void Physics(PythonModule<I3ConditionalModule> {lvalue},boost::shared_ptr<I3Frame>)
- icecube.gulliver_modules.fortytwo.isclose(a, b, rtol=1e-05, atol=1e-08, equal_nan=False)¶
Copied from a modern numpy distribution.
We are using this copy here because the build bots builds.icecube.wisc.edu are still using a very old numpy distribution which apparently does not know about the
isclose
function. Once they are updated, we cane remove this copy and just usenumpy.isclose
directly.Modifcation for integer types: just require a and b to be equal, ignore tolerances.