calibration/test_caf.py
=== stdout of calibration/tests/test_caf.py ===
[INFO] Steering file: /data/b2soft/buildbot/development/releases/current/calibration/tests/test_caf.py
[INFO] Gathering files for 'raw_physics' from input_files.json
[INFO] Found 2 files in 'raw_physics'
[INFO] Gathering files for 'raw_cosmic' from input_files.json
[INFO] Found 2 files in 'raw_cosmic'
[INFO] Gathering files for 'raw_beam' from input_files.json
[INFO] Found 0 files in 'raw_beam'
[WARNING] 'raw_beam' has no files associated with it!
[INFO] Getting values for 'expert_config' from the prompt script.
[INFO] Total number of raw-data files used as input = 4
[INFO] Requested use of Local backend
[INFO] Starting up new Pool with 4 processes
[INFO] Default CAF backend_args: {}
[INFO] Applying prompt settings to Calibration(name=KLMChannelStatus)
[INFO] Applying prompt settings to Collection(name=default)
[INFO] This appears to be a default Collection with no input files.
[INFO] Settings will be applied but it will likely be removed prior to the CAF process beginning.
[INFO] Applying prompt settings to Collection(name=raw)
[INFO] Adding Calibration(name=raw) to CAF process
[INFO] Checking for any invalid Collections in Calibrations.
[WARNING] Removing invalid Collection 'default' from Calibration 'KLMChannelStatus'.
[INFO] Initial status of KLMChannelStatus found to be state=init, iteration=0
[INFO] Attempting collector submission for calibration KLMChannelStatus.
[INFO] No File to IoV mapping required.
[INFO] Creating Job(KLMChannelStatus_raw_Iteration_0).Subjob(0)
[INFO] Creating Job(KLMChannelStatus_raw_Iteration_0).Subjob(1)
[INFO] Creating Job(KLMChannelStatus_raw_Iteration_0).Subjob(2)
[INFO] Creating Job(KLMChannelStatus_raw_Iteration_0).Subjob(3)
[INFO] No ArgumentsGenerator assigned to the MaxSubjobsSplitter so subjobs of Job(KLMChannelStatus_raw_Iteration_0) won't automatically have arguments assigned.
[INFO] MaxSubjobsSplitter created 4 Subjobs for Job(KLMChannelStatus_raw_Iteration_0)
[INFO] Submitting SubJob(KLMChannelStatus_raw_Iteration_0_0)
[INFO] Setting KLMChannelStatus_raw_Iteration_0_0 status to submitted
[INFO] SubJob(KLMChannelStatus_raw_Iteration_0_0) submitted
[INFO] Starting Sub-process: KLMChannelStatus_raw_Iteration_0_0
[INFO] stdout/err for subprocess KLMChannelStatus_raw_Iteration_0_0 visible at:
/tmp/tmpeho64zkb/calibration_results/KLMChannelStatus/0/collector_output/raw/0/stdout
/tmp/tmpeho64zkb/calibration_results/KLMChannelStatus/0/collector_output/raw/0/stderr
[INFO] Submitting SubJob(KLMChannelStatus_raw_Iteration_0_1)
[INFO] Setting KLMChannelStatus_raw_Iteration_0_1 status to submitted
[INFO] SubJob(KLMChannelStatus_raw_Iteration_0_1) submitted
[INFO] Submitting SubJob(KLMChannelStatus_raw_Iteration_0_2)
[INFO] Setting KLMChannelStatus_raw_Iteration_0_2 status to submitted
[INFO] SubJob(KLMChannelStatus_raw_Iteration_0_2) submitted
[INFO] Starting Sub-process: KLMChannelStatus_raw_Iteration_0_1
[INFO] stdout/err for subprocess KLMChannelStatus_raw_Iteration_0_1 visible at:
/tmp/tmpeho64zkb/calibration_results/KLMChannelStatus/0/collector_output/raw/1/stdout
/tmp/tmpeho64zkb/calibration_results/KLMChannelStatus/0/collector_output/raw/1/stderr
[INFO] Submitting SubJob(KLMChannelStatus_raw_Iteration_0_3)
[INFO] Setting KLMChannelStatus_raw_Iteration_0_3 status to submitted
[INFO] SubJob(KLMChannelStatus_raw_Iteration_0_3) submitted
[INFO] All requested jobs submitted.
[INFO] Starting Sub-process: KLMChannelStatus_raw_Iteration_0_2
[INFO] Starting Sub-process: KLMChannelStatus_raw_Iteration_0_3
[INFO] stdout/err for subprocess KLMChannelStatus_raw_Iteration_0_2 visible at:
/tmp/tmpeho64zkb/calibration_results/KLMChannelStatus/0/collector_output/raw/2/stdout
/tmp/tmpeho64zkb/calibration_results/KLMChannelStatus/0/collector_output/raw/2/stderr
[INFO] stdout/err for subprocess KLMChannelStatus_raw_Iteration_0_3 visible at:
/tmp/tmpeho64zkb/calibration_results/KLMChannelStatus/0/collector_output/raw/3/stdout
/tmp/tmpeho64zkb/calibration_results/KLMChannelStatus/0/collector_output/raw/3/stderr
[INFO] Setting KLMChannelStatus_raw_Iteration_0 status to submitted
[INFO] Calibration Machine KLMChannelStatus moved to state running_collector.
[INFO] Updating full collector job statuses.
[INFO] 0/4 Collector SubJobs finished in Calibration KLMChannelStatus Job KLMChannelStatus_raw_Iteration_0.
[INFO] Updating full collector job statuses.
[INFO] 0/4 Collector SubJobs finished in Calibration KLMChannelStatus Job KLMChannelStatus_raw_Iteration_0.
[INFO] Updating full collector job statuses.
[INFO] 0/4 Collector SubJobs finished in Calibration KLMChannelStatus Job KLMChannelStatus_raw_Iteration_0.
[INFO] Updating full collector job statuses.
[INFO] 0/4 Collector SubJobs finished in Calibration KLMChannelStatus Job KLMChannelStatus_raw_Iteration_0.
[INFO] Subprocess KLMChannelStatus_raw_Iteration_0_1 finished.
[INFO] Subprocess KLMChannelStatus_raw_Iteration_0_0 finished.
[INFO] Subprocess KLMChannelStatus_raw_Iteration_0_3 finished.
[INFO] Subprocess KLMChannelStatus_raw_Iteration_0_2 finished.
[INFO] Updating full collector job statuses.
[ERROR] Setting KLMChannelStatus_raw_Iteration_0_0 status to failed
[ERROR] Setting KLMChannelStatus_raw_Iteration_0_1 status to failed
[ERROR] Setting KLMChannelStatus_raw_Iteration_0_2 status to failed
[ERROR] Setting KLMChannelStatus_raw_Iteration_0_3 status to failed
[ERROR] Setting KLMChannelStatus_raw_Iteration_0 status to failed
[INFO] 4/4 Collector SubJobs finished in Calibration KLMChannelStatus Job KLMChannelStatus_raw_Iteration_0.
[INFO] Calibration Machine KLMChannelStatus moved to state collector_failed.
[INFO] Calibration Machine KLMChannelStatus moved to state failed.
[INFO] Printing summary of final CAF status.
=================================================
Calibrations Table
=================================================
name state checkpoint iteration
0 KLMChannelStatus failed init 0
=================================================
[INFO] End of CAF processing.
[INFO] Checking for failures...
[INFO] The BELLE2 environment variables are:
BELLE2_ANALYSES_PROJECT=belle2/analyses
BELLE2_ARCH=Linux_x86_64
BELLE2_BACKGROUND_DIR=/belle2/bkg
BELLE2_BACKGROUND_MIXING_DIR=/belle2/bkg.mixing
BELLE2_CONFIG_DIR=/belle2/config
BELLE2_DOWNLOAD=--ca-certificate=/cvmfs/belle.cern.ch/tools/certchain.pem https://software.belle2.org/download
BELLE2_EXAMPLES_DATA_DIR=/belle2/examples-data
BELLE2_EXCLUDE_PACKAGES=daq
BELLE2_EXTERNALS_DIR=/cvmfs/belle.cern.ch/el7/externals/v02-00-01
BELLE2_EXTERNALS_OPTION=opt
BELLE2_EXTERNALS_REPOSITORY=git@gitlab.desy.de:belle2/software/externals.git
BELLE2_EXTERNALS_SUBDIR=Linux_x86_64/opt
BELLE2_EXTERNALS_TOPDIR=/cvmfs/belle.cern.ch/el7/externals
BELLE2_EXTERNALS_VERSION=v02-00-01
BELLE2_GIT_SERVER=git@gitlab.desy.de:
BELLE2_MAX_BUILDS=8
BELLE2_NO_TOOLS_CHECK=1
BELLE2_OPTION=opt
BELLE2_PREPARE_PATH=/tmp/tmpaktpiljw
BELLE2_RELEASE=current
BELLE2_RELEASE_DIR=/data/b2soft/buildbot/development/releases/current
BELLE2_RUNDB_TOKEN=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJlbmRwb2ludHMiOlsicmVzdC92MS9ydW5zLyJdLCJuYW1lIjoiYmFzZjJfdW5pdHRlc3QifQ.QGfS2g5w0KwSCoL1BrLwyo-FwBzd6Gb60rbq036_as4
BELLE2_SOFTWARE_REPOSITORY=git@gitlab.desy.de:belle2/software/basf2.git
BELLE2_SUBDIR=Linux_x86_64/opt
BELLE2_TOOLS=/cvmfs/belle.cern.ch/tools
BELLE2_USER=b2soft
BELLE2_VALIDATION_DATA_DIR=/belle2/validation-data
BELLE2_VERSIONING_REPOSITORY=git@gitlab.desy.de:belle2/software/versioning.git
VO_BELLE2_SW_DIR=/data/b2soft/buildbot/development
[ERROR] Calibration failed, here is the submit.sh of the first collector job.
#!/bin/bash
# ---
# trap ctrl-c and call ctrl_c()
trap '(ctrl_c 130)' SIGINT
trap '(ctrl_c 143)' SIGTERM
function write_exit_code() {
echo "Writing $1 to exit status file"
echo "$1" > __BACKEND_CMD_EXIT_STATUS__
exit $1
}
function ctrl_c() {
trap '' SIGINT SIGTERM
echo "** Trapped Ctrl-C **"
echo "$1" > __BACKEND_CMD_EXIT_STATUS__
exit $1
}
# ---
if [ -z "${BELLE2_CONFIG_DIR}" ]; then
export BELLE2_CONFIG_DIR=/belle2/config
fi
if [ -z "${VO_BELLE2_SW_DIR}" ]; then
export VO_BELLE2_SW_DIR=/data/b2soft/buildbot/development
fi
if [ -z "${BELLE2_EXTERNALS_TOPDIR}" ]; then
export BELLE2_EXTERNALS_TOPDIR=/cvmfs/belle.cern.ch/el7/externals
fi
source /cvmfs/belle.cern.ch/tools/b2setup current
basf2 run_collector_path.py --job-information job_info.json
# ---
write_exit_code $?
[ERROR] Calibration failed, here is the stdout of the first collector job.
Belle II software tools set up at: /cvmfs/belle.cern.ch/tools
WARNING: Your preferred character encoding is not UTF-8.
-------> It is recommended to enable UFT-8 encoding, see 'man locale' for instructions.
Info: HOME environment variable is not set, therefore can not set JUPYTER_CONFIG_DIR to $HOME/.jupyter.
Environment setup for release: current
Central release directory : /data/b2soft/buildbot/development/releases/current
Warning: The release current is not supported any more. Please update to release-06-02-00
[INFO] Steering file: run_collector_path.py
[INFO] Job information file: job_info.json
[INFO] Using Global Tag Legacy_CollisionAxisCMS.
[INFO] Using Global Tag Legacy_IP_Information.
[INFO] Using Global Tag online_2020-08-11.
[INFO] Using Global Tag validation_2020-08-11.
[INFO] Using Global Tag patch_main_release-07.
[INFO] Using Global Tag patch_main_release-08.
[INFO] Using Global Tag patch_main_release-08_flip_refit.
[INFO] Modules and parameter settings in the path:
1. RootInput
inputFileNames=['file:///belle2/validation-data/calibration/cdst_e0008_r3121_hadronSkim/cdst_e0008_r3121_hadronSkim.root']
2. HistoManager
histoFileName=CollectorOutput.root
3. Gearbox
4. Geometry
5. TTDUnpacker
6. KLMUnpacker
7. KLMChannelStatusCollector
[INFO] Starting event processing, random seed is set to '133dfebbc577595993fd600e748e5a5f2c5961e1a6982fa49962cec3d7f05d50'
[INFO] Added file file:///belle2/validation-data/calibration/cdst_e0008_r3121_hadronSkim/cdst_e0008_r3121_hadronSkim.root
[INFO] Global tag override is in effect: input globaltags and default globaltag will be ignored
[INFO] Conditions Database: found working server
url = http://belle2db.sdcc.bnl.gov/b2s/rest/
[INFO] Conditions data: configured globaltags (highest priority first) are patch_main_release-08_flip_refit, patch_main_release-08, patch_main_release-07, validation_2020-08-11, online_2020-08-11, Legacy_IP_Information, Legacy_CollisionAxisCMS
[ERROR] Conditions Database: Problem parsing payload information.
globaltag = patch_main_release-08_flip_refit
server url = http://belle2db.sdcc.bnl.gov/b2s/rest/
error = Operation too slow. Less than 10240 bytes/sec transferred the last 60 seconds
[WARNING] Conditions data: Problem with payload metadata provider, trying to fall back to next provider...
provider = http://belle2db.sdcc.bnl.gov/b2s/rest/
[WARNING] TOPGeometry: old payload revision (before bugfix and update of optical properties) { module: Geometry }
**************************************************************
Geant4 version Name: geant4-11-01-patch-01 (10-February-2023)
Copyright : Geant4 Collaboration
References : NIM A 506 (2003), 250-303
: IEEE-TNS 53 (2006), 270-278
: NIM A 835 (2016), 186-225
WWW : http://geant4.org/
**************************************************************
[WARNING] ARICH geometry: no data available for merger 1 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 2 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 3 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 4 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 5 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 6 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 7 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 8 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 9 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 10 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 11 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 12 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[FATAL] 1 ERROR(S) occurred! The processing of events will not be started. { function: void Belle2::EventProcessor::process(const PathPtr&, long int) }
[INFO] ===Error Summary================================================================
[FATAL] 1 ERROR(S) occurred! The processing of events will not be started.
[ERROR] Conditions Database: Problem parsing payload information.
globaltag = patch_main_release-08_flip_refit
server url = http://belle2db.sdcc.bnl.gov/b2s/rest/
error = Operation too slow. Less than 10240 bytes/sec transferred the last 60 seconds
[WARNING] ARICH geometry: no data available for merger 12 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 11 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 10 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 9 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 8 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 7 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 6 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 5 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 4 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 3 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 2 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] ARICH geometry: no data available for merger 1 cooling body geometry. Cooling body will not be placed. { module: Geometry }
[WARNING] TOPGeometry: old payload revision (before bugfix and update of optical properties) { module: Geometry }
[WARNING] Conditions data: Problem with payload metadata provider, trying to fall back to next provider...
provider = http://belle2db.sdcc.bnl.gov/b2s/rest/
[INFO] ================================================================================
[ERROR] in total, 1 errors occurred during processing
[WARNING] in total, 14 warnings occurred during processing
WARNING - Attempt to delete the assembly store while geometry closed !
WARNING - Attempt to delete the region store while geometry closed !
WARNING - Attempt to delete the solid store while geometry closed !
WARNING - Attempt to delete the logical volume store while geometry closed !
WARNING - Attempt to delete the physical volume store while geometry closed !
Writing 1 to exit status file
[ERROR] Calibration failed, here is the stderr of the first collector job.
[FATAL] The test failed because an exception was raised (Command '['b2caf-prompt-run', 'Local', 'config.json', 'input_files.json', '--heartbeat', '20']' returned non-zero exit status 1.). Please re-run the build if this failure happened on bamboo. { function: }
[INFO] ===Error Summary================================================================
[FATAL] The test failed because an exception was raised (Command '['b2caf-prompt-run', 'Local', 'config.json', 'input_files.json', '--heartbeat', '20']' returned non-zero exit status 1.). Please re-run the build if this failure happened on bamboo.
[ERROR] Calibration failed, here is the stderr of the first collector job.
[ERROR] Calibration failed, here is the stdout of the first collector job.
[ERROR] Calibration failed, here is the submit.sh of the first collector job.
[INFO] ================================================================================
[ERROR] in total, 3 errors occurred during processing
=== stderr of calibration/tests/test_caf.py ===
TStreamerInfo::BuildCheck:0: RuntimeWarning:
The StreamerInfo of class Belle2::TRGTOPUnpackerStore read from file /belle2/validation-data/calibration/cdst_e0008_r3121_cosmicSkim/cdst_e0008_r3121_cosmicSkim.root
has the same version (=2) as the active class but a different checksum.
You should update the version to ClassDef(Belle2::TRGTOPUnpackerStore,3).
Do not try to write objects with the current class definition,
the files will not be readable.
TStreamerInfo::CompareContent:0: RuntimeWarning: One base class of the on-file layout version 2 and of the in memory layout version 2 for 'Belle2::TRGTOPUnpackerStore' is different: 'TObject' vs 'Belle2::RelationsInterface'
Traceback (most recent call last):
File "/data/b2soft/buildbot/development/releases/current/bin/Linux_x86_64/opt/b2caf-prompt-run", line 352, in
sys.exit(main())
File "/data/b2soft/buildbot/development/releases/current/bin/Linux_x86_64/opt/b2caf-prompt-run", line 346, in main
run_caf(cal_fw)
File "/data/b2soft/buildbot/development/releases/current/bin/Linux_x86_64/opt/b2caf-prompt-run", line 236, in run_caf
raise RuntimeError(f"{calibration.name} failed!")
RuntimeError: KLMChannelStatus failed!
=== end calibration/tests/test_caf.py ===