1. What’s New

This page explains the new features in the current release. This cannot cover all the changes in all packages but should help users to find out what needs to be adapted when changing to the new release.

1.1. Changes since release-05

Photons generated by PHOTOS in continuum events

Fixed the issue where PHOTOS photons were not correctly flagged in continuum events, e.g., charm decays (BII-5934). This was present in release-05-00-01 and earlier, including MC13 files.

1.1.1. Changes in the analysis package since release-05-00

Bremsstrahlung correction

  • Fixed that the copies of particles to which Bremsstrahlung was applied (whether this resulted in a added photon or not) are considered as sources of tracks. This allows to update the daughters in vertex fits that involve brems-corrected particles.

Variables

Modules

  • In TagVertex, all charged particles from the ROE are loaded to be included in the tag vertex fit (and not only those with a pion hypothesis)

  • The ParticleCombinerFromMC and the reconstructMCDecay function can set decayModeID extraInfo with an argument dmID. One can decide whether charge conjugated mode should be reconstructed with new boolean argument chargeConjugation (true by default).

  • The special treatment of MCMatching for the tau-decay is fixed. The treatment was working fine in release-04, but it was broken in the release-05.

1.1.2. Changes in the framework package since release-05-00-00

Conditions Database

  • b2conditionsdb-tag-merge has been added to merge a number of globaltags into a single globaltag in the order they are given. The result is equivalent to having multiple globaltags setup in the conditions access for basf2.

  • b2conditionsdb-tag-runningupdate has been added to calculate and apply the necessary updates to a running globaltag with a given staging globaltag.

  • b2conditionsdb-download has learned a new argument to clean all payload files not mentioned in the database file from the download area.

Tools

  • Added b2rundb-query to perform simple rundb queries from the command line.

Python Interface

  • The ANSIColors class has been added to the module terminal_utils to very easily add color output to the terminal.

  • Added a RunDB class which makes writing scripts using the rundb more straightforward.

1.2. Changes since release-04

Neutral hadrons from ECLClusters get momentum from the cluster energy

Since release-04 it has been possible to load ECLClusters under the neutral hadron hypothesis. Previously we assumed a mass when calculating the particle momentum, however this leads to problems when, for example, a \(K_L^0\) deposits less than its mass energy in the ECL. This happens about 50% of the time.

The momentum of neutral hadrons from the ECL is now set to the clusterE.

Bremsstrahlung correction

The BremsFinder module has been developed to find relations between tracks and photons that are likely to have been emitted by these tracks via Bremsstrahlung. The matching quality figure of merit is based on the angular distance between the photon ECL cluster and the extrapolated hit position of the track at the ECL. The function correctBrems performs the actual correction. There is also a reimplementation of Belle’s Bremsstrahlung correction approach of looking for photons in a cone around tracks (correctBremsBelle), which is recommended for b2bii analyses.

Warning

While it is technically possible to perform a TreeFit after applying Bremsstrahlung correction, the fit performance is unfortunately quite bad. However, there is already an improvement in the pipeline that should fix this issue. It will probably be available in one of the next light releases.

MC reconstruction and MC matching

The ParticleCombinerFromMC module and its corresponding wrapper function reconstructMCDecay should be used instead of findMCDecay to reconstruct decay modes based on MC information.

The DecayStringGrammar has been extended with new exception markers for Bremsstrahlung, decay in flight, and misidentification.

Exceptions for the MC matching of daughter particles with the DecayStringGrammar are propagated to the mother particle.

Redefinition of angle variables

The kinematic variables decayAngle, daughterAngle and pointingAngle now return the angle instead of its cosine.

Protection of ParticleLists and particle combinations

It is no longer allowed to use the label "all" for a particle list if a cut is applied. Reconstructed decays need to preserve electric charge. However, this can be deactivated if you know what you are doing, e.g. in searches for New Physics.

1.2.1. Changes in the analysis package since release-04-02

Warning

Global includes like from basf2 import * are almost completely removed from the analysis package.

Vertex Fitting

Warning

The convenience function for the TreeFitter module has been renamed from vertexTree to vertex.treeFit().

Warning

The KFit convenience functions have been merged to the new function vertex.kFit(). One of its input arguments is fit_type, which specifies whether you want to perform a mass fit (fit_type=mass), a vertex fit (fit_type=vertex), a mass-constrained vertex fit (fit_type=massvertex), or a vertex fit with the mother constrained to the beam four-momentum (fit_type=fourC).

  • Added smearing parameter to vertex.kFit. When you perform a vertex fit with an IP tube constraint (fit_type=vertex and constraint=iptube), the IP profile is smeared by the given value. The IP tube is defined by a neutral track pointing towards the boost direction starting from the smeared IP profile. Setting smearing=0 gives the original iptube option, which is an elongated ellipsoid in boost direction.

  • Fixed TreeFitter bias on V0s by propagating the correct TrackFitResult from V0 daughters (see BII-6336).

  • Added FeedthroughParticle and InternalTrack to vertex.treeFit. This now provides compatibility of TreeFitter with the bremsstrahlung recovery modules. In principle, TreeFitter can now be used for decays for which Bremsstrahlung correction has been applied. However, there are massive performance issues. Do not blindly use TreeFit but cross-check your results with KFit. Further improvements are planned.

Standard Particle Lists

  • The list label "all" is now forbidden from use with cuts. This can introduce some very subtle bugs in user scripts. The following code will no longer work:

    fillParticleList("gamma:all", "clusterE > 0.02", path=mypath)

    Instead you should replace it with a meaningful name of your choice. For example:

    fillParticleList("gamma:myMinimumThresholdList", "clusterE > 0.02", path=mypath)

  • Added stdHyperons.py, this analysis script reconstructs the standard hyperons Xi-, Xi0, and Omega-.

  • The default vertex fitter for the V0 lists stdKshorts and stdLambdas has been changed from Rave to TreeFit. However, a new option called fitter has been added that allows to change the vertex fitter back to raveFit or to kFit. This change is motivated by a much faster execution time (see BII-5699) and a compatible performance as summarized here.

  • Added a new standard list stdCharged.stdMostLikely which creates 5 mutually-exclusive lists of track-based particles under their most likely hypothesis.

  • Moved the KlId check from ParticleLoader to stdKlongs. This modification does not change the definition of the K_L0:all list and, in general, of the user-defined \(K_{L}^0\) lists.

  • Removed the functions mergedKshorts and mergedLambdas, which returned the standard V0 lists anyway.

  • Removed the deprecated pi0 lists pi0:loose and pi0:veryloose

  • stdPi0s and stdPhotons updated to match Jan2020 recommendations. Naming scheme updated to indicate date of optimization.

Variables

Note

All variables will return a quietNaN instead of -999 or other out-of-the-sky numbers in case of errors. To study the failures use the meta variable ifNANgiveX, which replaces nan with a value of your choice.

Warning

The variables decayAngle, daughterAngle and pointingAngle now return an angle instead of a cosine.

Note

This is not the generator-level match (see BII-5741) but simply ignores photons added by the BremsFinder.

Hint

It is recommended to use variablesToExtraInfo prior to a vertex fit to access pre-fit values, not only for mass significances but also for vertex positions, etc.

  • Fixed truth kinematic variables (mcPX, mcPY, …) to respect the current reference frame, so that they can be accessed not only in the lab frame but for example as well in the CMS frame using useCMSFrame.

  • Extended functionality of recoil kinematics variables (pxRecoil, pyRecoil, …) to be usable with any requested frame.

  • Added isFromV0 to identify particles that come from tracking-level V0’s (as opposed to those created at analysis-level using modularAnalysis.reconstructDecay).

  • Removed isExtendedSignal whose exact behavior is mirrored by isSignalAcceptWrongFSPs.

  • Added the new variable collection event_kinematics which contains all variables available after running the EventKinematics module.

  • Added unmask, which unmasks certain indicated bits in a variable (set certain bits in a variable to zero).

  • Added a new variable util create_isSignal_alias which creates customized isSignal, accepting certain match errors.

  • Removed the previously deprecated variable clusterHypothesis.

  • Added firstCDCLayer, returns the ID of the CDC layer closest to the IP with a hit attributed to the particles track

  • Added the new variable collection cluster_average. It contains the cluster variables that can only be executed for particles with daughters, currently maxWeightedDistanceFromAverageECLTime and weightedAverageECLTime.

  • Added a variable for TDCPV resolution study DeltaTRes and legacy Belle DeltaTBelle.

  • Variable TagVmcLBoost renamed to mcTagVLBoost and variable TagVmcOBoost renamed to mcTagVOBoost

Modules

  • When merging ParticleLists using the ParticleListManipulator (modularAnalysis.copyLists) duplicates (not multiple candidates) are removed on a first-come, first-served basis, i.e. the order of the input particles matters. Now, the new parameter variable can be provided to determine which duplicate should be preferred. The logic is the same as for the BestCandidateSelection module. Furthermore, duplicates, which differ only by the order of their daughter particles, are now identified.

  • The ParticleLoader can create neutron particle lists now. ECL cluster with the hadron hypothesis as well as KLM cluster are used as source.

  • Tweaks to the ParticleLoader for how it creates particles from V0s, it now preserves the link to the mdst object. These changes are all under-the-hood and require no actions by users.

  • In TreeFitter bugfix in mass constraint: chi2 correctly calculated now.

  • In TreeFitter change of initialisation values increases performance in all fits with tracks.

  • In TreeFitter max iterations set to 100, increases efficiency slightly in many cases.

  • Removed module CreateBsigFromFullRecoBtag

  • In BtubeCreator module, functionality to associate Btube to user’s chosen B is added.

  • Removed module FSRCorrection in favour of BremsFinder and BelleBremRecovery.

  • In ChargedPidMVA module, remove handling of particle list modified by the removed FSRCorrection module.

  • In ChargedPidMVA module, apply MVA category cuts retrieved from the DB payload.

  • Added ChargedPidMVAMulticlass module to allow performing global PID (one-vs-rest), using an MVA trained in multi-class mode.

  • Added a BremsFinder to unpack the bremsstrahlung relations provided by tracking.

Hint

This tool is the recommended way to correct for bremsstrahlung photons in Belle II analyses.

  • Modified writePi0EtaVeto function and added oldwritePi0EtaVeto function. The weight files used in release-03 were not available in release-04. The latter uses old weight files optimized with MC9, while new weight files, which are optimized with MC12, are used in writePi0EtaVeto

  • Enabled to select daughters which will be used to perform fitting with KFit. One can use the selector ^ to select daughters.

  • KFit can be used to fit vertices of fully-inclusive particles, for which it ignores the daughters without defined p-value.

  • RestOfEventBuilder, EventKinematics and EventShapeCalculator modules can use most likely charged particle hypothesis according to PID information.

  • Added the ParticleCombinerFromMC module and the reconstructMCDecay to find and create ParticleList from a given DecayString. They can be used instead of the MCDecayFinder and findMCDecay which are not fully tested and maintained.

  • In TagVertex added Btube as a possible constraint for the tag B vertex fit. The useFitAlgorithm parameter in the module is replaced by constraintType (which can be set to tube) and trackFindingType. All computations done in the module are now using double precision. The module is also updated to allow for the computations of new variables related to the tracks used in the tag vertex fit.

  • Added a InclusiveDstarReconstruction to inclusively reconstruct D* mesons by estimating the four vector using slow pions.

  • In the TagV function, added the possibility to use KFit instead of Rave to perform the tag vertex fit.

  • Both flavor taggers support ROE masks. Added new maskName argument.

  • Added new DecayString grammar for c_AddedRecoBremsPhoton, c_DecayInFlight, and c_MisID. If one uses ?addbrems in own DecayString, c_AddedRecoBremsPhoton is ignored, thus isSignal works as isSignalAcceptBremsPhotons. One can add (decay) and/or (misID) to the beginning of the particle name to accept c_DecayInFlight and/or c_MisID. The following is an example, 'reconstructDecay('D0:sig -> (misID)K-:loose (decay)pi+:loose', '', path=mypath)'

  • Modified modularAnalysis.matchMCTruth to work always recursively. Once one calls modularAnalysis.matchMCTruth for a particle, the MC Matching is done for not only the particle but also all daughters of the particle correctly. This modification does not change the MC Matching of the particle for which modularAnalysis.matchMCTruth is called.

  • Added feature to add multiple TTrees to the same output file via the VariablesToEventBasedTree module.

  • Rest Of Event can be created using MCParticles using modularAnalysis.buildRestOfEventFromMC

  • Removed -->, =>, and ==> from the list of allowed arrow types for the DecayString.

  • In TagVertex module, added the possibility to use the truth information from the tag particles in the vertex fit. The option useTruthInFit = True switches that on.

  • In TagVertex module, implemented an internal change: tag particles are loaded as Particle object from the ROE and not anymore as TrackFitResult. This should have no effect to users.

  • Added a warning to the ParticleCombiner module for decay strings violating electric charge conservation which can be turned off by setting allowChargeViolation=True.

  • In TagVertex module, added the possibility to perform vertex fit with the tag particle tracks rolled back to their primary vertex points. The option useRollBack = True switches that on.

  • Added an argument to the ParticleCombiner module that allows to deactivate the automatic reconstruction of the charge-conjugated mode. In reconstructDecay the option is called chargeConjugation, which is True by default.

  • The MC matching with the DecayString properly works in hierarchical decay. For example,

    from modularAnalysis import reconstructDecay, matchMCTruth
reconstructDecay('B0:signal      -> mu+ mu- ... ?gamma',   '', path=mypath)
reconstructDecay('B0:generic     -> D*-:Dpi pi+:all',      '', path=mypath)
reconstructDecay('Upsilon(4S):BB -> B0:generic B0:signal', '', path=mypath)
matchMCTruth('Upsilon(4S)', path=mypath)

    In the above case, missing daughters (massive FSP and gamma) of B0:signal are accepted for not only B0:signal but also Upsilon(4S), so that isSignal can be 1. Another example,

    reconstructDecay('D-:pipi0 -> pi-:all pi0:all',      '', path=mypath)
reconstructDecay('B0:Dpi   -> D-:pipi0 pi+:all ...', '', path=mypath)
matchMCTruth('B0:Dpi', path=mypath)

    In this case, one wants to accept missing massive daughters in B0:Dpi decay but not in D-:pipi0 decay. So, if the decay of D-:pipi0 in the MC truth level is D- -> pi+ pi- pi- pi0, isSignal of D-:pipi0 and B0:Dpi will be 0, since there are missing daughters in D-:pipi0 decay. If one wants to accept missing daughters in D-:pipi0, please use the DecayString grammar in the reconstruction of D-:pipi0 or use isSignalAcceptMissing variable instead of isSignal.

Utilities and core objects

  • A set of functions DistanceTools has been added to compute distances between (straight) tracks and vertices.

  • Added a relation between Track-based Particles and TrackFitResults.

  • Added a getTrackFitResult method to Particle.

  • Added a function to retrieve electric charge of a particle based on its pdg code.

  • Added a RotationTools.h file with few functions related to rotation

Tutorials and Examples

  • Added a tutorial about creating aliases (examples/VariableManager/variableAliases.py)

1.2.2. Changes in the analysis package since release-04-00

Variables

  • Now vertex variables dr, dx, dy, dz, dphi, dcosTheta... take into account the nontrivial transformation of track parameters relative to IP when used for tracks. dr did not center at zero before and now it does.

  • Added isDescendantOfList and isMCDescendantOfList meta variables that allow to check whether the particle is the descendant of the list (or if it is matched to the descendant of the list) at any generation. The variables have recursive search inside, they extend isDaughterOfList and isGrandDaughterOfList.

  • Added mcParticleIsInMCList which checks the underlying MC particles (either matched or generator level).

  • Fixed bug and renamed mcFlavorOfOtherB0 variable to mcFlavorOfOtherB, which accepts now neutral and charged B candidates.

  • Removed clusterCRID which duplicates clusterConnectedRegionID.

  • Fixed goodBelleLambda and now it returns extraInfo(goodLambda) on Belle data.

  • Allow use of meta variables in creation of aliases by replacing non alphanumeric characters with underscores in the alias name

  • Modified daughterAngle to accept generalized variable indeces instead of simple indeces. A generalized index is a column-separated string of daughter indexes belonging to different generations, starting from the root particle. For example, 0:2:1 indicates the second daughter (0) of the third daughter (2) of the first daughter (0) of the particle. Of course, conventional indexes are still working as expected: 1 still indicates the second daughter of the particle.

  • Added daughterCombination, that returns a variable calculated on the sum of an arbitrary number of daughters. Generalized indexes are supported. This variable is mostly intended to calculate the invariant mass or the recoil mass of any set of particles belonging to a decay tree.

  • Fixed isSignal, which always accepted c_MissingResonance and c_MissFSR/c_MissPHOTOS even if one used =direct=>, =norad=>, or =exact=>. Now it correctly respects the decay string grammar. The other isSignal* variables, such as isSignalAcceptMissing, are also fixed.

  • Added useAlternativeDaughterHypothesis, that returns a variable re-calculated using alternative mass assumptions for the particle’s daughters.

  • Restructured the mc_flight_info collection by removing the non-sense error variables and replacing mc_flightTime and mc_flightDistance, which were aliases created using matchedMC, with the dedicated variables mcFlightTime and mcFlightDistance, respectively.

  • Removed output_variable option in the Deep Flavor Tagger and introduced a standard variable DNN_qrCombined for the output. The new variable returns the flavor times the dilution factor as the category-based Flavor Tagger output variables FBDT_qrCombined and FANN_qrCombined do. Now we can evaluate the output of both taggers in the same way.

  • Bug fix to guard against range exceptions in KSFWVariables (see BII-6138).

Modules

  • In TreeFitter fixed a bug in life time calculation. A constant in the jacobian was missing. As a result the propagated error was slightly overestimated.

  • Fix to nested RestOfEvent objects (see BII-5649)

  • Fixed bugs in MCDecayFinder and findMCDecay. Inefficiency and large background when one used =direct=> or sub-decay such as D*+ -> [D0 -> pi+ pi- pi0] pi+ are fixed. But still the module has some bugs if one uses K_S0.

Conditions DB

  • In ChargedPidMVAWeights payload class, added MVA category cut strings in basf2-compliant format.

Full Event Interpretation

  • Addition of hadronic FEI channels involving baryons. This includes the addition to default channels the following particles: p, Lambda_c+, Sigma+ and Lambda0. Baryonic modes must be switched on when calling particles = fei.get_default_channels(baryonic = True) using a flag (baryonic = True).

1.2.3. Changes in the framework package since release-04-00-00

Conditions Database

  • b2conditionsdb: Conditions DB interface has been optimized to work with larger globaltags.

  • b2conditionsdb-diff and b2conditionsdb-iov by default don’t show any database internal ids anymore but those can be reenabled with --show-ids

  • b2conditionsdb-dump has learned a new argument to show the content of a payload valid for a given run in a given globaltag

  • There are new python classes to handle iovs, see conditions_db.iov

Logging

Miscellaneous

  • Added support for writing udst files as output from BGxN N>0 files in light releases (see BII-3622). This means skimming is fully supported with a light release.

  • Added support for b2bii in light releases. However, this comes at the cost of no longer being able to convert ExtHits and ECLHits.

  • The RootInput will now by default skip events which have an error flag set. This can be changed with the discardErrorEvents parameter.

  • Added the function basf2.get_file_metadata to quickly obtain the FileMetaData object of a given basf2 output file.

  • Added the tools b2code-sphinx-build and b2code-sphinx-warnings to build the sphinx documentation or check for warnings when building the documentation.

1.2.4. Changes in the decfiles package since release-04-02

Note

The decfiles directory was part of the analysis package until release-03-02. For documentation on earlier changes, please refer to the comments at the top of DECAY_BELLE2.DEC.

Changes to DECAY_BELLE2.DEC

Note

All changes listed below are documented in BELLE2-NOTE-PH-2020-008 - please refer to this note for a more detailed and specific description of updates.

  • Updates to charm hadron decays

    • Substantial changes to D0, anti-D0, D+, D-, D_s+, and D_s- decay tables

    • Changes to charm baryon decay tables

  • Other updates

    • Added decay table for sigma_0 particle, previously missing from decay file

1.2.5. Changes in the b2bii package since release-04-01

Warning

b2biiConversion.setupB2BIIDatabase() will be deprecated soon.

convertBelleMdstToBelleIIMdst

  • Added a switch to deactivate nisKsFinder.

  • Renamed applyHadronBJSkim to applySkim, and added switches to deactivate HadronA and HadronB skims.

  • Added a switch to deactivate converting RecTrg_Summary3 table.

Conversion

  • Removed convertECLCrystalEnergies() and convertExtHits() in the conversion.

  • Added mass-constraint fit information of pi0:mdst, including chiSquared, ndf and pValue as extraInfo().

  • Added m_final(3) in RecTrg_Summary3 table as eventExtraInfo(rectrg_summary3_m_final).

1.3. Changes since release-03

Removal of default analysis path and NtupleTools

Warning

The default path (“analysis_main”) and the NtupleTools are now removed.

This is a major backward-compatibility breaking change. Please update your user scripts to create your own path (basf2.create_path) and to use the variable manager tools (such as VariablesToNtuple).

If your previously working example script from release-03 looked something like this:

from basf2 import *
from stdCharged import stdPi
from modularAnalysis import *
stdPi("good")
ntupleFile("myFile.root") # <-- now removed
ntupleTree("pi+:good", ['pi+', 'Momentum']) # <-- now removed
process(analysis_main)
print(statistics)

You should update it to this:

import basf2 # better not to import all
from stdCharged import stdPi
from modularAnalysis import variablesToNtuple
mypath = basf2.Path() # create your own path (call it what you like)
stdPi("good", path=mypath)
variablesToNtuple("pi+:good", ['px', 'py', 'pz', 'E'], path=mypath)
basf2.process(mypath)
print(basf2.statistics)

See also

The example scripts available here:

$BELLE2_RELEASE_DIR/analysis/examples/VariableManager

Switch of beam spot information from nominal to measured values.

The interaction point position and its uncertainties are now taken from the database with values provided by the tracking group. All beam kinematics information is also moved to the database, which will eventually be measured on data. For now they are the values provided by the accelerator.

Warning

The previous definition of the boost included a small rotation to align it with the HER. This is no longer possible with the new structure. The definition of CMS is therefore slightly changed. The impact should be at the percent level.

If you have a physics analysis sensitive to this change: please discuss with the software / performance groups and add a comment to BII-4360.

See also

The beam information can be accessed with Ecms, beamPx, beamPy, beamPz, and beamE.

Note

As a consequence of this, there is now a difference between the variables ( dx, dy, dz ) compared to ( x, y, z ); and similarly for variables ( mcDecayVertexX, mcDecayVertexY, mcDecayVertexZ ) compared to ( mcDecayVertexFromIPX, mcDecayVertexFromIPY and mcDecayVertexFromIPZ ).

Redesign of the Conditions Database Interface

The configuration and handling of the connection to the conditions database has been completely rewritten in a more coherent and modular way. We now have a new and consistent configuration interface, global tag replay and advanced checks: If users specify a global tag to be used which is either marked as invalid in the database or which cannot be found in the database the processing is now aborted. See Conditions Database Overview for details.

Restrict usage of useDB=False for Geometry creation

Creating the geometry from XML files instead of the configuration in the Database may lead to wrong results. So while the option useDB=False is still necessary to debug changes to the geometry definitions it is now restricted to only be used for exp, run = 0, 0 to protect users from mistakes.

This also changes the behavior of add_simulation() and add_reconstruction(): If a list of components is provided this will now only change the digitization or reconstruction setup but will always use the full geometry from the database.

Loading ECLClusters under multiple hypotheses

It is now possible to load \(K_L^0\) particles from clusters in the ECL. This has several important consequences for the creation of particles and using combinations containing \(K_L^0\) s or other neutral hadrons in the analysis package. This is handled correctly by the ParticleLoader and ParticleCombiner (the corresponding convenience functions are modularAnalysis.fillParticleList and modularAnalysis.reconstructDecay). Essentially: it is forbidden from now onwards for any other analysis modules to create particles.

Deprecated RAVE for analysis use

The (external) RAVE vertex fitter is not maintained. Its use in analysis is therefore deprecated. We do not expect to remove it, but do not recommend its use for any real physics analyses other than benchmarking or legacy studies.

Instead we recommend you use either KFit (vertex.kFit) for fast/simple fits, or TreeFit (vertex.treeFit) for more complex fits and fitting the full decay chain. Please check the Tree Fitter pages for details about the constraints available. If you are unable to use TreeFitter because of missing functionality, please submit a feature request!

Warning

The default fitter for vertex.fitVertex has been changed to KFit.

Tidy up and rename of Helicity variables.

Renamed helicity variables in the VariableManager following consistent logic. We added the new variable cosAcoplanarityAngle.

Warning

cosHelicityAngle is now cosHelicityAngleMomentum, and cosHelicityAngle has a new definition (as in the PDG 2018, p. 722).

Old name

New name

cosHelicityAngle

cosHelicityAngleMomentum

cosHelicityAnglePi0Dalitz

cosHelicityAngleMomentumPi0Dalitz

cosHelicityAngleIfCMSIsTheMother

cosHelicityAngleBeamMomentum

New DecayStringGrammar for custom MCMatching

Users can use new DecayStringGrammar to set properties of the MCMatching. Then isSignal, mcErrors and other MCTruthVariables behave according to the property.

Once DecayStringGrammar is used with reconstructDecay, users can use isSignal instead of several specific variables such as isSignalAcceptMissingNeutrino. If one doesn’t use any new DecayStringGrammar, all MCTruthVariables work same as before.

The grammar is useful to analyze inclusive processes with both fully-inclusive-method and sum-of-exclusive-method. There are also new helper functions genNMissingDaughter and genNStepsToDaughter to obtain the detailed MC information.

You can find examples of usage in Marker of unspecified particle, Grammar for custom MCMatching.

1.3.1. Changes in the analysis package

TreeFitter

  • Fix the \(\phi\)-dependent loss of performance for displaced vertices (BII-4753).

Flavor Tagger

  • Default Expert (testing mode) does not create repositories and does not save weight files locally. It only loads the payloads directly from the database using the database payload names as mva identifiers.

  • BtagToWBosonVariables adapted to work with the new ROE.

  • All release validation and performance evaluation scripts added to $BELLE2_RELEASE_DIR/analysis/release-validation/CPVTools.

  • The flavor tagger creates and adds default aliases into the collection list flavor_tagging.

Vertex Fitting

  • Added IP tube constraint to KFit.

  • The parameter confidenceLevel of the ParticleVertexFitter now always rejects the particle candidates with a p-value lower than the specified one. Specifically, setting confidenceLevel to 0 does not reject candidates with p-value equal to 0 anymore. Thus, the meaning of this parameter is now the same as for the TreeFitter.

FEI

  • Removed the backward compatibility layer (pid_renaming_oktober_2017). Only FEI trainings from release-02 are supported. Please update to FEIv4_2018_MC9_release_02_00_01 or newer.

Variables

Warning

we overhauled the helicity variables and added new ones to replace the NtupleHelicityTool. We renamed cosHelicityAngle to cosHelicityAngleMomentum, cosHelicityAnglePi0Dalitz to cosHelicityAngleMomentumPi0Dalitz, and cosHelicityAngleIfCMSIsTheMother to cosHelicityAngleBeamMomentum. We added the variables cosHelicityAngle and cosAcoplanarityAngle defining them as the PDG 2018, p. 722.

  • The function to create variable aliases for particles selected from a decay string has been fixed and extended to allow better customization (BII-4437)

    See also

    variables.utils.create_aliases_for_selected()

  • Added a variable that returns the distance between a KLMCluster and its associated Track klmClusterTrackDistance

  • Access to tag vertexing quality indicators for resolution function study, TagVNDF, TagVChi2, and TagVChi2IP.

  • Statistically correct combination of p-values pValueCombination

  • Added the meta variable clusterHasFailedTiming to check if the ECL cluster’s timing fit failed.

  • Added the meta variable clusterHasFailedErrorTiming to check if the calculation of the ECL cluster’s error on the timing failed.

  • The chi2 and ndf from KFit vertices are stored as extraInfo for debugging.

  • Added trackFitHypothesisPDG which returns the PDG code of the successfull track fit hypothesis that was used to create this particle (that might be different from the PDG code of the particle list)

  • Added variables for 2D Armenteros-Podolanski plot, ArmenterosLongitudinalMomentumAsymmetry, ArmenterosDaughter1Qt, and ArmenterosDaughter2Qt

  • Fall-back value changed from zero to NaN for several track variables.

  • Added sourceObjectIsInList which checks whether there are particles from the same underlying track or cluster in a given particle list. Similar to isInList but a tighter check.

  • Added isCloneOfSignalSide and hasAncestorFromSignalSide variables to monitor the clone particle contamination in ROE. These variables use generator information.

  • The variables for the fast reco HLT trigger stage have been removed, as we are currently not planning to have it in the early data taking.

  • The event-based variable KLMEnergy is removed. It is completely meaningless (BII-5002).

  • The variables ( dx, dy, dz, dr, dphi, dcosTheta ) are now with respect to the interaction point (as documented). This now makes them different with respect to the cartesian position ( x, y, z ) with respect to the origin.. This is similarly true for ( mcDecayVertexFromIPX, mcDecayVertexFromIPY, mcDecayVertexFromIPZ ) versus ( mcDecayVertexX, mcDecayVertexY and mcDecayVertexZ ).

  • Added the meta variable eclClusterSpecialTrackMatched to check if any track related to an ECL cluster satisfies certain criteria. It can be used to apply an individual track-cluster matching.

  • Added missing error variables DeltaZErr and DeltaBoostErr. Fixed bug in TagV error functions.

  • Replaced pValue (tracking variable) with chiProb (vertexing variable) in the collection of vertexing variables.

  • Replaced pValue (always the track-fit pvalue) with chiProb (context-dependent \(\chi^2\) probability variable) in the vertexing variable collection.

  • Added the meta variables averageValueInList and medianValueInList which calculate the arithmetic mean and median of a given variable in a given particle list.

  • Removed nECLClusters (BII-4476). The number of ECLClusters is not a good measure for anything except the age of the input mdst file. Recent changes to the handling of ECLClusters and their hypotheses have changed this (already arbitrary) number to another (equally arbitrary but different) number.

    Hint

    For most common use cases, the recommended alternative is the total number of photons (unique photon hypothesis clusters) in the event:

    from variables import variables
from stdPhotons import stdPhotons
stdPhotons("cdc", path=mypath) # creates the all and cdc lists
variables.addAlias("nPhotonsAll", "nParticlesInList(gamma:all)")
variables.addAlias("nPhotonsCDC", "nParticlesInList(gamma:cdc)")
variables.addAlias("totalPhotonEECL", "totalEnergyOfParticlesInList(gamma:all)")

  • Deprecated clusterHypothesis following changes to the ECLCluster storage on mdst. Please use clusterHasNPhotons and clusterHasNeutralHadron.

  • Removed ClusterHadronIntensity and clusterUniqueID.

  • Added helix extrapolation from charged tracks.

  • Added clusterMdstIndex variable to get ECLCluster mdst index for particles that do not originate from ECLCluster.

  • Added a convenience M2, that returns the invariant mass squared of a given Particle (determined from the particle’s 4-momentum vector).

  • Added eclClusterOnlyInvariantMass, that returns the invariant mass of the particles, computed with cluster information

  • Added grandDaughterDiffOf, grandDaughterDiffOfPhi, and grandDaughterDiffOfClusterPhi. They behave like daughterDiffOf, but use grandDaughters’ information rather than daughters’.

  • Renamed ClusterHasPulseShapeDiscrimination to clusterHasPulseShapeDiscrimination, PulseShapeDiscriminationMVA to clusterPulseShapeDiscriminationMVA, and ClusterNumberOfHadronDigits to :b2:var`clusterNumberOfHadronDigits`.

  • Removed the misleading variable McFlavorOfTagSide.

  • Added a variable mcFlavorOfOtherB which returns the tag-side flavor by checking the other Upsilon4S daughter without using the ROE. This variable is part of the default flavor_tagging collection.

  • The name of the variable MCTagBFlavor changed to internalTagVMCFlavor. It should be used only by experts for debugging.

  • Added MC flight information variables, mcFlightDistance and mcFlightTime

Modules

  • Added the new module AllParticleCombiner which is also available via the function modularAnalysis.combineAllParticles. It creates a new Particle as the combination of all unique Particles from the passed input ParticleLists.

  • The ParticleLoader can load Rest Of Event as a particle using a new function modularAnalysis.fillParticleListFromROE. This ROE particle can be combined with any other particle or written down using usual variables.

    Another option is to load missing momentum as a particle by supplementing useMissing = True argument to the function, mentioned above.

  • Fixed a bug in the BestCandidateSelection module: When allowMultiRank=True there always at least two candidates with rank of one, even if they didn’t have the same variable value (BII-4460)

    This affects all users of modularAnalysis.rankByLowest() and modularAnalysis.rankByHighest() if they passed allowMultiRank=True

  • Fixed a bug in the BestCandidateSelection module: now the numBest parameter works as expected.

  • Removal of the unsupported ECLClusterInfoModule.

  • Added vertex.fitPseudo function and a pseudo vertex fitting module to add a covariance matrix when a vertex fit is not possible. E.g. for \(\pi^0\) decays.

  • Added the new module SignalSideVariablesToDaughterExtraInfo. It adds ExtraInfo to a specified particle (typically a daughter on the signal side). The corresponding information is calculated in the RestOfEvent so it is supposed to be executed only in for_each ROE paths.

  • Fixed a bug and extended the functionality of the RestOfEventBuilder module. When providing a ParticleList of composite particles as an additional source for building the ROE, the composite particles are now decomposed and their final state daughters are added to the ROE (unless they are part of the signal side or already present in the ROE). Previously, composite particles were not decomposed and the first composite particle of the first ParticleList of composite particles (and only this one) was always added to the ROE.

  • The module VertexFitUpdateDaughters now always updates the daughters (as advertised).

Modular Analysis

  • A new boolean argument has been added to the function modularAnalysis.buildRestOfEvent. It is called belle_sources and should be switched to True (default is False) if you are analyzing converted Belle MC or data. It triggers the ROE to be constructed from all tracks and from Belle’s specific gamma:mdst list.

  • Added signal region function modularAnalysis.signalRegion. By default, this function enables a new variable isSignalRegion and excludes the defined area when processing is done on data.

Standard Particle Lists

  • Updated V0 lists. The standard list became the merged list, a combined list of particles coming from V0 objects merged with a list of particles combined using the analysis ParticleCombiner module.

    stdV0s.stdKshorts returns a Ks list called K_S0:merged stdV0s.stdLambdas returns a Lambda list called Lambda0:merged

    mergedKshorts() and mergedLambdas() are now deprecated, and return the standard lists.

  • The definition of the standard V0 lists slightly changed. For Lambdas, the modularAnalysis.markDuplicate() function is now used to detect duplicates among the same list (V0 and ReconstructDecay), rather than between different lists. For Ks, the modularAnalysis.markDuplicate() function is no longer used.

  • Updated V0 lists by making them explicitly call the appropriate vertex fit.

  • Fixed a bug in the merged Lambda list, which used an incorrect mass window.

  • Fix the stdKlongs.stdKlongs lists.

  • Updated charged standard PID cut values to reflect the correct efficiencies. This recovers the efficiency loss reported in BIIDP-1065.

Tutorials

  • Fix B2A202 to load Klongs as advertised.

  • Fix B2A801 to use JPsiKs as signal channel and to show how to run on Belle data/MC. Central database global tag updated.

1.3.2. Changes in the framework package

Job Information File

basf2 has a new command line parameter --job-information=<filename> to create a json file with some statistics on the processing and detailed information on all created output files.

See also

BELLE2-NOTE-TE-2017-012: Software and Computing Interface

Python Interface

  • Additional utility function b2test_utils.check_error_free used to automate a code quality (or indeed any) tool check. It requires execution without any output as part of the unit tests. For example the analysis package now requires no doxygen or cppcheck warnings.

  • Additional utility function b2test_utils.configure_logging_for_tests to improve reproducibility of log output for testing purposes by replacing certain setup dependent names of directories with placeholders.

  • The Pager class has been moved to a new module terminal_utils which now also contains a new class InputEditor to open a editor when asking the user for a long message similar to git commit

  • The compatibility imports of sys, os and the fw object have been removed from basf2. Every method which was previously part of the fw object is now just a function in the basf2 module.

Core Framework

  • Addition of helpful StoreArray::hasRelationTo and StoreArray::hasRelationFrom methods.

  • Deprecated static methods to register StoreArray instances have been removed.

Command line Tools

  • b2file-metadata-add will now only update the file in the FileCatalog if it is already registered. It will also now correctly remove the old LFN from the file catalog.

  • b2file-merge will by default no longer register files in the file catalog. You can either supply --add-to-catalog as argument or run b2file-catalog-add on the output file. Use the latter if you want to write scripts which work also with older releases.

  • b2file-normalize has been added to remove timestamps and similar non-reproducible features from ROOT files.

  • b2file-remove-branches has been added to remove obsolete branches from old files.

  • the commands of the b2conditionsdb tool are now also available with dashed version, e.g. b2conditionsb-tag-show

  • A number of old tool names which were kept for compatibility have been removed. The old tool names still exist but raise an error pointing to the new name.

Modules

  • b2:mod:RootOutput module: changed default value for updateFileCatalog to False. This will avoid creating Belle2FileCatalog.xml or warning about overwriting entries in the file catalog by default. The file catalog is only needed when loading parent files and can always be created later using b2file-catalog-add

    See also

    b2file-catalog-add

1.3.3. Changes in the ecl package

Modules

  • Remove all getHypothesisId() calls and logic by a new logic that allows clusters to have multiple flags to indicate their hypothesis. This is used to remove duplicate clusters that are identical under different hypothesis.

1.4. Changes since release-02-01

Moved to C++17

The whole software including the ROOT in the externals is now compiled using the C++17 standard. This should not affect many users but there are a few rare cases where this might lead to compilation problems of analysis code as some deprecated features have been removed. The most notable are

  • throw(TypeName) exception specifiers, just remove them.

  • std::auto_ptr which should be replaced by std::unique_ptr

  • some older parts of the <functional> header.

In particular if you compile a standalone program that links against the ROOT in the Belle2 externals this now also needs to be compiled in C++17 mode. You can do this by adding -std=c++17 to the compiler arguments.

Note

It’s best to directly pass the output of root-config --cflags to the compiler. That way you always pass the correct flags needed for the particular ROOT version setup.

Build system moved to Python3

This is a major update of SCons but most users should not notice any difference except for two instances:

  • If you update an existing working directory from an older release you might run into an error

    scons: *** [...] TypeError : intern() argument 1 must be string, not unicode
scons: building terminated because of errors.
TypeError: intern() argument 1 must be string, not unicode:
[...]

    In this case please remove all .scon* files in the top level of your software directory and rerun scons

  • In the unlikely case that you have custom SConscript files which are not Python 3 compatible you will have to update them.

1.4.1. Changes in the analysis package

Variables

There have been improvements and additions to the variables available in the variable manager. Some older, unhelpful, or deprecated variables have been removed, but this should be less than previous major releases. As usual, please ask at https://questions.belle2.org if anything is unclear or your favourite variable seems to have been removed. There is likely a good reason.

New Modular Analysis Tools

Fitters

  • Many improvements to TreeFitter, which is now the recommended vertex fitter almost all use cases, even for simple vertices with two tracks. Please refer to the TreeFitter documentation for full details.

    See also

    This TreeFitter tutorial (October 2018).

  • Bug fixes to OrcaKinFit.

  • KFit is now accessible from other basf2 modules.

Loading V0 Particles

  • When loading V0 particles (i.e. \(K_S^0\), \(\Lambda^0\), or converted photons) using the ParticleLoader (fillParticleList) you must now specify the daughters in a decay string. For example, to load \(\Lambda^0\to p^+\pi^-\) decays from V0s:

    from modularAnalysis import fillParticleList
fillParticleList('Lambda0 -> p+ pi-', '0.9 < M < 1.3', path=mypath)

Tutorials and Examples

  • The style of many of the tutorial scripts have been updated to assist with and provide examples of these new changes. See:

    $BELLE2_RELEASE_DIR/analysis/examples/tutorials

1.4.2. Changes in the framework package

Python Interface

The basf2 python interface has been restructured into multiple files to ease maintenance. For the user this should be transparent. Some changes the user might notice are

  • There is now a new utility function basf2.find_file to allow looking for files in the release directory or separate examples or validation directories.

  • There is now an automatic Jupyter integration: Calling basf2.process in a Jupyter notebook should now automatically run the processing in a separate process and show a nice progress bar.

  • The obsolete “fw” object has been deprecated and all functionality which previously was accessed using basf2.fw.* is now directly accessible as basf2.*. If you use basf2.fw you will get a deprecation warning.

  • When using from basf2 import * the sys and os packages were also silently imported and available in the current script. This has been deprecated. In general, using import * is not recommended but if you have to use it and use the sys or os module in the script please make sure you import them yourself after from basf2 import *

  • Display and colouring of log messages in Jupyter has been significantly improved and should now be much faster.

  • There’s a new implementation for pager.Pager which can also show output from C++ programs and will display the output incrementally instead of waiting for all output before showing everything.

Command Line Tools

  • b2file-check now supports files with zero events correctly.

  • b2file-merge now checks that real/MC data flag is consistent for all input files and refuse to merge mixed real/MC files.

  • The subcommands iov and diff of b2conditionsdb have improved output and learned the new option --human-readable to convert the IntervalOfValidity numbers to easier to read strings.

  • There is a new sub command dump for b2conditionsdb to dump the contents of a payload file in a human readable form on the terminal for quick inspection.

  • There is a new command b2conditionsdb-extract which allows to convert a payload to a TTree with one entry per requested run number. This allows to easily monitor how payloads change over the course of time.

  • There is a new command b2conditionsdb-recommend which will recommend users a global tag to use when processing a given input file.

  • There is a new command b2conditionsdb-request to allow requesting the inclusion of locally prepared database payload into the official global tags.

Core Framework

  • Environment::isMC() is now available to consistently distinguish between real and MC data. To use it in C++ please use

    #include <framework/core/Environment.h>
bool isMC = Environment::Instance().isMC();

    and for use in python

    from ROOT import Belle2
isMC = Belle2.Environment.Instance().isMC();

  • We have now a new prototype for advanced multi processing using ZMQ. It is disabled by default but can be activated using

    from ROOT import Belle2
env = Belle2.Environment.Instance().setUseZMQ(True)

    See also

    Pull request [PR#2790]

  • There is now support for named relations to allow multiple relations between the same pair of StoreArray.

Core Modules

  • The ProgressBar module will now notice if the output is written to a log file instead of the terminal and behave accordingly which should clean up logfiles considerably when this module is used.

  • The RootOutput module now allows to split output files after a certain file size is reached using the outputSplitSize parameter.

    Warning

    This will set the amount of generated events stored in the file metadata to zero as it is not possible to determine which fraction ends up in which output file.

    Also the user can now choose the compression algorithm to choose either no compression, zlib, LZMA or LZ4 compression. LZ4 is a newer compression standard with slightly worse compression than zlib but much faster decompression speed.

  • removed CrashHandler module

  • removed FileLogger module

Conditions Database:

  • Database objects are now immutable (const) to prevent accidental modification of conditions data.

  • The old and outdated fallback database in framework/data/database.txt has been removed. If you still set this by hand in your steering file your script will fail. Please use /cvmfs/belle.cern.ch/conditions if you really have to set a fallback database manually.

  • The RunInfo database object which is supposed to contain all necessary information about each run now has support to contain the trigger pre-scale information.

Logging System

  • We now have “Log message variables” which allow to send the same log message with varying content. This greatly helps with filtering log messages as it allows to group messages which have the same content and just differ in their variables. In C++ they can be used by adding a LogVar instance to the output,

    B2INFO("This is a log message" << LogVar("number", 3.14) << LogVar("text", "some text"));

    while in python the variables can be given as additional keyword arguments,

    basf2.B2INFO("This is a log message", number=3.14, text="some text")

    In both cases the names of the variables can be chosen feely and the output should be something like

    [INFO] This is a log message
        number = 3.14
        text = some text

    See also

    Log Variables

  • The logging system is now able to send its message to python sys.stdout objects to allow intercepting log messages in python. To enable please set basf2.logging.enable_python_logging to True. This is automatically enabled when running inside of a jupyter notebook.

  • Log messages can also be formatted as json objects where each log message will be printed as a one line json object to allow parsing of logfiles using scripts.

    See also

    basf2.logging.add_json()

Utilities

  • We have a new and advanced formula parser implementation in the framework package. It manages to handle ** correctly as in python and now allows using normal parenthesis for grouping operations in addition to square brackets.

    See also

    formula variable

  • We now provide a variety of RAII scope guards to free or restore a resource or value when the guard object goes out of scope. For example to make sure a variable is reset to is original value one could use

    #include <framework/utilities/ScopeGuard.h>

int main() {
  int myValue{5};
  {
    auto guard = Belle2::ScopeGuard::guardValue(myValue, 32);
    // now myValue is 32
  }
  // now myValue is reverted to 5 independently of how the scope
  // is left (normal, return statement, exception)
}

    We provide convenience functions to guard simple values, pointer deletion, output stream flags and the current working directory but the interface is general enough that almost anything can be guarded by this ScopeGuard object.

  • There is now RootFileManager to allow multiple modules to write to the same root output file. It will take care to open the file when the first module requests it and close it when the last module is finished with the root file. The primary use case is for the VariablesToNtuple and similar modules to allow having multiple ntuples or trees in the same root file.

  • We now have a implementation of the c++17 std::visit overloaded pattern in framework/utils/Utils.h called Belle2::Utils::VisitOverload.