b2conditionsdb: Conditions DB interface
Contents
4.6. b2conditionsdb
: Conditions DB interface#
This tool provides a command line interface to all the tasks related to the Conditions database: manage globaltags and iovs as well as upload new payloads or download of existing payloads.
The usage of this tool is similar to git: there are sub commands like for
example tag
which groups all actions related to the management of
globaltags. All the available commands are listed below.
Users need a valid JSON Web Token (JWT) to authenticate to the conditions database when using this tool. For practical purposes, it is only necessary to know that a JWT is a string containing crypted information, and that string is stored in a file. More informations about what a JWT is can be found on Wikipedia.
The tool automatically queries the JWT issuing server
(https://token.belle2.org) and gets a valid token by asking the B2MMS username
and password. The issued “default” JWT has a validity of 1 hour; after it
expires, a new JWT needs to be obtained for authenticating the conditions
database. When retrieved by this tool, the JWT is stored locally in the file
${HOME}/b2cdb_${BELLE2_USER}.token
.
Some selected users (e.g. the calibration managers) are granted a JWT with an
extended validity (30 days) to allow smooth operations with some automated
workflows. Such “extended” JWTs are issued by a different server
(https://token.belle2.org/extended). B2MMS username and password are necessary
for getting the extended JWT. The extended JWT differs from the default JWT
only by its validity and can be obtained only by manually querying the
alternative server. If queried via web browser, a file containing the extended
JWT will be downloaded in case the user is granted it. The server can also be
queried via command line using
wget --user USERNAME --ask-password --no-check-certificate https://token.belle2.org/extended
or curl -u USERNAME -k https://token.belle2.org/extended
.
If the environment variable ${BELLE2_CDB_AUTH_TOKEN}
is defined and points
to a file containing a valid JWT, the b2conditionsdb
tools use this token
to authenticate with the CDB instead of querying the issuing server. This is
useful when using an extended token. Please note that, in case the JWT to which
${BELLE2_CDB_AUTH_TOKEN}
points is not valid anymore, the
b2conditionsdb
tools will attempt to get a new one and store it into
${BELLE2_CDB_AUTH_TOKEN}
. If a new extended token is needed, it has to be
manually obtained via https://token.belle2.org/extended and stored into
${BELLE2_CDB_AUTH_TOKEN}
.
usage: b2conditionsdb [-h] [--debugging] [--help-full] [--base-url BASE_URL]
[--auth-token AUTH_TOKEN]
{diff,download,dump,iov,iovs,legacydownload,tag,upload}
...
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).
4.6.1. Sub-command b2conditionsdb diff
#
This command allows to compare two globaltags. It will show the changes in a format similar to a unified diff but by default it will not show any context, only the new or removed payloads. Added payloads are marked with a
+
in the first column, removed payloads with a-
If
--full
is given it will show all payloads, even the ones common to both globaltags. The differences can be limited to a given run and limited to a set of payloads names using--filter
or--exclude
. If the--regex
option is supplied the search term will be interpreted as a python regular expression where the case is ignored.Changed in version release-03-00-00: modified output structure and added
--human-readable
Changed in version after: release-04-00-00 added parameter
--checksums
and--show-ids
b2conditionsdb diff [-h] [--debugging] [--help-full] [--base-url BASE_URL]
[--auth-token AUTH_TOKEN] [--full] [--run N N]
[--human-readable] [--checksums] [--show-ids]
[--run-range FIRST_EXP FIRST_RUN FINAL_EXP FINAL_RUN]
[-f SEARCHTERM] [-e EXCLUDE] [-r]
TAGNAME1 TAGNAME2
Required Arguments
- TAGNAME1
base for comparison
- TAGNAME2
tagname to compare
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).- --full
If given print all iovs, also those which are the same in both tags
- --run
exp and run numbers to limit showing iovs to a ones present in a given run
- --human-readable
If given the iovs will be written in a more human friendly format. Also repeated payload names will be omitted to create a more readable listing.
- --checksums
If given don’t show the revision number but the md5 checksum
- --show-ids
If given also show the payload and iov ids for each iov
- --run-range
Can be four numbers to limit the run range to be comparedOnly iovs overlapping, even partially, with this range will be considered.
- -f, --filter
only payloads matching this pattern will be included. Pattern is case insensitive
- -e, --exclude
payloads matching this pattern will be excluded. Pattern is case insensitive
- -r, --regex
if given, –filter or –exclude options will be interpreted as a python regular expression (see https://docs.python.org/3/library/re.html)
4.6.2. Sub-command b2conditionsdb download
#
This command allows to download the information from one or more globaltags from the central database to be used locally.
The command requires at least one tag name to download. It will check for existing payloads in the output directory and only download payloads which are not present or don’t have the expected checksum.
By default this script will create a local directory called conditions/
which contains a metadata.sqlite
with all the payload information of all
selected globaltags and sub directories containing all the payload files.
This can be changed by specifying a different name for the metadata file
using the -o
argument but the payloads will always be saved in sub
directories in the same directory as the sqlite file.
Changed in version release-04-00-00: Previously this command was primarily intended to download payloads for
one globaltag and optionally create a text file with payload information
as well as download all necessary file. This has been changed and will
now create a sqlite file containing the payload metadata. If you need the
old behavior please use the command b2conditionsdb-legacydownload
b2conditionsdb download [-h] [--debugging] [--help-full] [--base-url BASE_URL]
[--auth-token AUTH_TOKEN] [-o DATABASEFILE] [-f]
[--append] [--no-download | --only-download]
[--delete-extra-payloads] [--ignore-missing]
[-j NPROCESS] [--retries RETRIES]
[--tag-pattern | --tag-regex]
[TAGNAME [TAGNAME ...]]
Required Arguments
- TAGNAME
globaltag to download
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).- -o, --dbfile
Name of the database file to create (default: “conditions/metadata.sqlite”)
- -f, --force
Don’t ask permission if the output database file exists
- --append
Append to the existing database file if possible. Otherwise the content in the database file will be overwritten
- --no-download
Don’t download any payloads, just fetch the metadata information
- --only-download
Assume the metadata file is already filled, just make sure all payloads are downloaded
- --delete-extra-payloads
if given this script will delete all extra files that follow the payload naming convention
AB/{name}_r{revision}.root
if they are not referenced in the database file.- --ignore-missing
Ignore missing globaltags and download all other tags
- -j
Number of concurrent connections to use for file download (default: 1)
- --retries
Number of retries on connection problems (default: 3)
- --tag-pattern
if given, all globaltags which match the shell-style pattern TAGNAME will be downloaded:
*
stands for anything,?
stands for a single character.- --tag-regex
if given, all globaltags matching the regular expression given by TAGNAME will be downloaded (see https://docs.python.org/3/library/re.html).
4.6.3. Sub-command b2conditionsdb dump
#
New in version release-03-00-00.
This command will dump the payload contents stored in a given payload. One
can either specify the payloadId (from a previous output of
b2conditionsdb iov
), the payload name and its revision in the central
database, or directly specify a local database payload file.
Examples
Dump the content of a previously downloaded payload file:
$ b2conditionsdb dump -f centraldb/dbstore_BeamParameters_rev_59449.root
Dump the content of a payload by name and revision directly from the central database:
$ b2conditionsdb dump -r BeamParameters 59449
Dump the content of the payload by name which is valid in a given globaltag for a given experiment and run:
$ b2conditionsdb dump -g BeamParameters main_2021-08-04 0 0
Or directly by payload id from a previous call to b2conditionsdb iov
:
$ b2conditionsdb dump -i 59685
Usage
Depending on whether you want to display a payload by its id in the
database, its name and revision in the database or from a local file
provide one of the arguments -i
, -r
, -f
or -g
Changed in version after: release-04-00-00
added argument -r
to directly dump a payload valid for a given run
in a given globaltag
b2conditionsdb dump [-h] [--debugging] [--help-full] [--base-url BASE_URL]
[--auth-token AUTH_TOKEN] ([-i PAYLOADID | -r NAME
REVISION | -f FILENAME | -g NAME GLOBALTAG EXP RUN)]
[--show-typenames] [--show-streamerinfo]
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).- -i, --id
payload id to dump
- -r, --revision
Name and revision of the payload to dump
- -f, --file
Dump local payload file
- -g, --valid
Dump the payload valid for the given exp, run number in the given globaltag
- --show-typenames
If given show the type names of all classes. This makes output more crowded but can be helpful for complex objects.
- --show-streamerinfo
If given show the StreamerInfo for the classes in the the payload file. This can be helpful to find out which version of a payload object is included and what are the members
4.6.4. Sub-command b2conditionsdb iov
#
This command lists all IoVs defined in a given globaltag. The list can be limited to a given run and optionally searched using –filter or –exclude. If the –regex option is supplied the search term will be interpreted as a python regular expression where the case is ignored.
Changed in version release-03-00-00: modified output structure and added --human-readable
Changed in version after: release-04-00-00
added parameter --checksums
and --show-ids
Changed in version after: release-08-00-04
added parameter --run-range
b2conditionsdb iov [-h] [--debugging] [--help-full] [--base-url BASE_URL]
[--auth-token AUTH_TOKEN] [--run N N] [--detail]
[--human-readable] [--checksums] [--show-ids]
[--run-range FIRST_EXP FIRST_RUN FINAL_EXP FINAL_RUN]
[-f SEARCHTERM] [-e EXCLUDE] [-r]
TAGNAME
Required Arguments
- TAGNAME
globaltag for which the the IoVs should be listed
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).- --run
exp and run numbers to limit showing iovs to a ones present in a given run
- --detail
if given show a detailed information for all IoVs including details of the payloads
- --human-readable
If given the iovs will be written in a more human friendly format. Also repeated payload names will be omitted to create a more readable listing.
- --checksums
If given don’t show the revision number but the md5 checksum
- --show-ids
If given also show the payload and iov ids for each iov
- --run-range
Can be four numbers to limit the run range to be shownOnly iovs overlapping, even partially, with this range will be shown.
- -f, --filter
only payloads matching this pattern will be included. Pattern is case insensitive
- -e, --exclude
payloads matching this pattern will be excluded. Pattern is case insensitive
- -r, --regex
if given, –filter or –exclude options will be interpreted as a python regular expression (see https://docs.python.org/3/library/re.html)
4.6.5. Sub-command b2conditionsdb iovs
#
This command allows to modify, delete or copy iovs of a globaltag. If no other command is given do nothing.
b2conditionsdb iovs [-h] [--debugging] [--help-full] [--base-url BASE_URL]
[--auth-token AUTH_TOKEN]
{copy,delete,modify} ...
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).
Sub-command b2conditionsdb iovs copy
#
This command allows to copy the iovs from a globaltags to another one, optionally limiting the iovs to be copied to those of a specific payload, revision, IoV id or run range.
b2conditionsdb iovs copy [-h] [--debugging] [--help-full]
[--base-url BASE_URL] [--auth-token AUTH_TOKEN]
[--iov-id IOV_ID] [--iov-pattern IOV_PATTERN]
[--run-range FIRST_EXP FIRST_RUN FINAL_EXP FINAL_RUN]
[--fully-contained] [--set-run-range] [-f SEARCHTERM]
[-e EXCLUDE] [-r] [--revision revision] [--dry-run]
[--replace] [-j NPROCESS]
INPUT_TAGNAME OUTPUT_TAGNAME
Required Arguments
- INPUT_TAGNAME
globaltag for which the the IoVs should be copied
- OUTPUT_TAGNAME
globaltag to which the the IoVs should be copied
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).- --iov-id
IoVid of the iov to be considered
- --iov-pattern
whitespace-separated string with pattern of the iov to be considered. Use * to mark the fields that shold be ignored. Valid patterns are 0 0 -1 -1 (a very specific IoV), 0 * -1 -1 (any iov that starts in any run of exp 0 and ends exactly in exp -1, run -1), * * 3 45 (any Iov ending in exp 3, run 45, regardless from where it starts).
- --run-range
Can be four numbers to limit the run range to be considered Only iovs overlapping, even partially, with this range will be considered.
- --fully-contained
If given together with –run_range limit the list of payloads to the ones fully contained in the given run range
- --set-run-range
If given together with –run_range modify the interval of validity of partially overlapping iovs to be fully contained in the given run range
- -f, --filter
only payloads matching this pattern will be included. Pattern is case insensitive
- -e, --exclude
payloads matching this pattern will be excluded. Pattern is case insensitive
- -r, --regex
if given, –filter or –exclude options will be interpreted as a python regular expression (see https://docs.python.org/3/library/re.html)
- --revision
Specify the revision of the payload to be removed
- --dry-run
Don’t do anything, just print what would be done
- --replace
Modify the iovs in the output tag to avoid overlaps
- -j
Number of concurrent threads to use.
Sub-command b2conditionsdb iovs delete
#
This command allows to delete the iovs from a globaltags, optionally limiting the iovs to be deleted to those of a specific payload, revision, IoVid or run range.
b2conditionsdb iovs delete [-h] [--debugging] [--help-full]
[--base-url BASE_URL] [--auth-token AUTH_TOKEN]
[--iov-id IOV_ID] [--iov-pattern IOV_PATTERN]
[--run-range FIRST_EXP FIRST_RUN FINAL_EXP FINAL_RUN]
[--fully-contained] [-f SEARCHTERM] [-e EXCLUDE]
[-r] [--revision revision] [--dry-run]
[-j NPROCESS]
INPUT_TAGNAME
Required Arguments
- INPUT_TAGNAME
globaltag for which the the IoVs should be deleted
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).- --iov-id
IoVid of the iov to be considered
- --iov-pattern
whitespace-separated string with pattern of the iov to be considered. Use * to mark the fields that shold be ignored. Valid patterns are 0 0 -1 -1 (a very specific IoV), 0 * -1 -1 (any iov that starts in any run of exp 0 and ends exactly in exp -1, run -1), * * 3 45 (any Iov ending in exp 3, run 45, regardless from where it starts).
- --run-range
Can be four numbers to limit the run range to be considered Only iovs overlapping, even partially, with this range will be considered.
- --fully-contained
If given together with –run_range limit the list of payloads to the ones fully contained in the given run range
- -f, --filter
only payloads matching this pattern will be included. Pattern is case insensitive
- -e, --exclude
payloads matching this pattern will be excluded. Pattern is case insensitive
- -r, --regex
if given, –filter or –exclude options will be interpreted as a python regular expression (see https://docs.python.org/3/library/re.html)
- --revision
Specify the revision of the payload to be removed
- --dry-run
Don’t do anything, just print what would be done
- -j
Number of concurrent threads to use.
Sub-command b2conditionsdb iovs modify
#
This command allows to modify the iovs from a globaltags, optionally limiting the iovs to be modified to those of a specific payload, revision, IoV id or run range.
b2conditionsdb iovs modify [-h] [--debugging] [--help-full]
[--base-url BASE_URL] [--auth-token AUTH_TOKEN]
[--iov-id IOV_ID] [--iov-pattern IOV_PATTERN]
[--run-range FIRST_EXP FIRST_RUN FINAL_EXP FINAL_RUN]
[--fully-contained] [-f SEARCHTERM] [-e EXCLUDE]
[-r] [--revision revision] [--dry-run]
[-j NPROCESS]
INPUT_TAGNAME NEW_IOV
Required Arguments
- INPUT_TAGNAME
globaltag for which the the IoVs should be modified
- NEW_IOV
New iov to be set to all considered iovs. It should be a string with 4 numbers separated by spaces. Use * to mark the fields that should not be modified. For example if 7 0 * * is given the iov (7, 1, 9, 42) will become (7 0 9 42).
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).- --iov-id
IoVid of the iov to be considered
- --iov-pattern
whitespace-separated string with pattern of the iov to be considered. Use * to mark the fields that shold be ignored. Valid patterns are 0 0 -1 -1 (a very specific IoV), 0 * -1 -1 (any iov that starts in any run of exp 0 and ends exactly in exp -1, run -1), * * 3 45 (any Iov ending in exp 3, run 45, regardless from where it starts).
- --run-range
Can be four numbers to limit the run range to be considered Only iovs overlapping, even partially, with this range will be considered.
- --fully-contained
If given together with –run_range limit the list of payloads to the ones fully contained in the given run range
- -f, --filter
only payloads matching this pattern will be included. Pattern is case insensitive
- -e, --exclude
payloads matching this pattern will be excluded. Pattern is case insensitive
- -r, --regex
if given, –filter or –exclude options will be interpreted as a python regular expression (see https://docs.python.org/3/library/re.html)
- --revision
Specify the revision of the payload to be removed
- --dry-run
Don’t do anything, just print what would be done
- -j
Number of concurrent threads to use.
4.6.6. Sub-command b2conditionsdb legacydownload
#
This command allows to download a globaltag from the central database to be used locally, either as lookup directory for payloads or as a standalone local database if –create-dbfile is specified.
The command requires the TAGNAME to download and optionally an output directory which defaults to centraldb in the local working directory. It will check for existing payloads in the output directory and only download payloads which are not present or don’t have the expected checksum.
One can filter the payloads to be downloaded by payload name using the –filter, –exclude and –regex options.
New in version release-04-00-00: This has been renamed from download
and is kept for compatibility
Warning
Downloading a globaltag should be done in the new format creating sqlite database files. Please use this legacy tool only for downloading “small” globaltags or very few payloads.
b2conditionsdb legacydownload [-h] [--debugging] [--help-full]
[--base-url BASE_URL] [--auth-token AUTH_TOKEN]
[-c] [-f SEARCHTERM] [-e EXCLUDE] [-r]
[-j NPROCESS] [--retries RETRIES]
[--run-range FIRST_EXP FIRST_RUN FINAL_EXP FINAL_RUN]
[--tag-pattern | --tag-regex]
TAGNAME [DIR]
Required Arguments
- TAGNAME
globaltag to download
- DIR
directory to put the payloads into (default: “centraldb”)
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).- -c, --create-dbfile
if given save information about all payloads in DIR/database.txt
- -f, --filter
only payloads matching this pattern will be included. Pattern is case insensitive
- -e, --exclude
payloads matching this pattern will be excluded. Pattern is case insensitive
- -r, --regex
if given, –filter or –exclude options will be interpreted as a python regular expression (see https://docs.python.org/3/library/re.html)
- -j
Number of concurrent connections to use for file download (default: 1)
- --retries
Number of retries on connection problems (default: 3)
- --run-range
Can be four numbers to limit the run range to be downloadedOnly iovs overlapping, even partially, with this range will be downloaded.
- --tag-pattern
if given, all globaltags which match the shell-style pattern TAGNAME will be downloaded:
*
stands for anything,?
stands for a single character. If -c is given as well the database files will beDIR/TAGNAME.txt
- --tag-regex
if given, all globaltags matching the regular expression given by TAGNAME will be downloaded (see https://docs.python.org/3/library/re.html). If -c is given as well the database files will be
DIR/TAGNAME.txt
4.6.7. Sub-command b2conditionsdb tag
#
This command allows to list, show, create modify or clone globaltags in the central database. If no other command is given it will list all tags as if “b2conditionsdb tag show” was given.
b2conditionsdb tag [-h] [--debugging] [--help-full] [--base-url BASE_URL]
[--auth-token AUTH_TOKEN]
{clone,create,list,merge,modify,runningupdate,show,state}
...
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).
Sub-command b2conditionsdb tag clone
#
This command allows to clone a given globaltag with a new name but still containing all the IoVs defined in the original globaltag.
b2conditionsdb tag clone [-h] [--debugging] [--help-full]
[--base-url BASE_URL] [--auth-token AUTH_TOKEN]
TAGNAME NEWNAME
Required Arguments
- TAGNAME
globaltag to be cloned
- NEWNAME
name of the cloned globaltag
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).
Sub-command b2conditionsdb tag create
#
This command creates a new globaltag in the database with the given name and description. The name can only contain alpha-numeric characters and the characters ‘+-_:’.
b2conditionsdb tag create [-h] [--debugging] [--help-full]
[--base-url BASE_URL] [--auth-token AUTH_TOKEN]
[-u USER]
TYPE TAGNAME DESCRIPTION
Required Arguments
- TYPE
type of the globaltag to create, usually one of DEV or RELEASE
- TAGNAME
name of the globaltag to create
- DESCRIPTION
description of the globaltag
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).- -u, --user
username who created the tag. If not given we will try to supply a useful default
Sub-command b2conditionsdb tag list
#
This command allows to list all globaltags, optionally limiting the output to ones matching a given search term. By default invalidated globaltags will not be included in the list, to show them as well please add –with-invalid as option. Alternatively one can use –only-published to show only tags which have been published
If the –regex option is supplied the search term will be interpreted as a python regular expression where the case is ignored.
b2conditionsdb tag list [-h] [--debugging] [--help-full] [--base-url BASE_URL]
[--auth-token AUTH_TOKEN] [--detail]
[--with-invalid | --only-published] [-f SEARCHTERM]
[-e EXCLUDE] [-r]
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).- --detail
show detailed information for all tags instead of summary table
- --with-invalid
if given also invalidated tags will be shown
- --only-published
if given only published tags will be shown
- -f, --filter
only tags matching this pattern will be included. Pattern is case insensitive
- -e, --exclude
tags matching this pattern will be excluded. Pattern is case insensitive
- -r, --regex
if given, –filter or –exclude options will be interpreted as a python regular expression (see https://docs.python.org/3/library/re.html)
Sub-command b2conditionsdb tag merge
#
This command allows to merge a number of globaltags into a single globaltag. Payloads from later globaltags in the list of arguments are used to fill gaps present in earlier globaltags.
The result is equivalent to having multiple globaltags setup in the conditions access for basf2 (highest priority goes first).
- Warning:
The order of the globaltags is highest priority first, so payloads from globaltags earlier on the command line will be taken with before globaltags from later tags.
This command requires that all globaltags are overlap free.
For each globaltag in the list we copy all payloads to the output globaltag if there is no payload of that name valid for the given interval of validity in any previous globaltags in the list.
If the payload overlaps partially with a payload from a previous globaltag in the list the interval of validity is shortened (and possibly split) to not overlap but to just fill the gaps.
For example:
globaltag A contains
payload1, rev 2, valid from 1,0 to 1,10
payload1, rev 3, valid from 1,20 to 1,22
payload2, rev 1, valid from 1,0 to 1,-1
globaltag B contains
payload1, rev 1, valid from 1,1 to 1,30
payload2, rev 2, valid from 0,1 to 1,20
Then running b2conditionsdb tag merge -o output A B
, the output globaltag
after the merge will contain:
payload1, rev 2, valid from 1,0 to 1,10
payload1, rev 1, valid from 1,11 to 1,19
payload1, rev 3, valid from 1,20 to 1,22
payload1, rev 1, valid from 1,23 to 1,30
payload2, rev 2, valid from 0,1 to 0,-1
payload2, rev 1, valid from 1,0 to 1,-1
When finished, this command will print a table of payloads and their
validity and from which globaltag they were taken. If --dry-run
is given
it will only print the list of payloads.
Optionally one can specify --run-range
to limit the run range for which
the merging should take place. In the example above, running with
--run-range 1 0 1 21
the result would be
payload1, rev 2, valid from 1,0 to 1,10
payload1, rev 1, valid from 1,11 to 1,19
payload1, rev 3, valid from 1,20 to 1,21
payload2, rev 1, valid from 1,0 to 1,21
New in version release-05-01-00.
b2conditionsdb tag merge [-h] [--debugging] [--help-full]
[--base-url BASE_URL] [--auth-token AUTH_TOKEN] -o
OUTPUT [--dry-run]
[--run-range FIRST_EXP FIRST_RUN FINAL_EXP FINAL_RUN]
[-j NPROCESS]
globaltag [globaltag ...]
Required Arguments
- globaltag
name of the globaltag
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).- --dry-run
Don’t do anything, just print a table with the results
- --run-range
Can be for numbers to limit the run range to putin the output globaltag: All iovs will be limited to be in this range.
- -j
Number of concurrent threads to use for creating payloads into the output globaltag.
required named arguments
- -o, --output
Name of the output globaltag
Sub-command b2conditionsdb tag modify
#
This command allows to change the name or description of an existing globaltag. You can supply any combination of -n,-d,-t and only the given values will be changed
b2conditionsdb tag modify [-h] [--debugging] [--help-full]
[--base-url BASE_URL] [--auth-token AUTH_TOKEN]
[-n NAME] [-d DESCRIPTION] [-t TYPE] [-u USER]
[-s STATE]
TAGNAME
Required Arguments
- TAGNAME
globaltag to modify
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).- -n, --name
new name
- -d, --description
new description
- -t, --type
new type of the globaltag
- -u, --user
username who created the tag. If not given we will try to supply a useful default
- -s, --state
new globaltag state, see the command
tag state
for details
Sub-command b2conditionsdb tag runningupdate
#
This command will calculate and apply the necessary updates to a running globaltag with a given staging globaltag
Running tags are defined as “immutable for existing data but conditions for newer runs may be added” and the only modification allowed is to add new payloads for new runs or close existing payloads to no longer be valid for new runs.
This command takes previously prepared and validated payloads in a staging globaltag and will then calculate which payloads to close and what to add to the running globaltag.
For this to work we require
A running globaltag in the state “RUNNING”
A (experiment, run) number from which run on the update should be valid. This run number needs to be
bigger than the start of validity for all iovs in the running tag
bigger than the end of validity for all closed iovs (i.e. not valid to infinity) in the running tag
A staging globaltag with the new payloads in state “VALIDATED”
payloads in the staging tag starting at (0,0) will be interpreted as starting at the first valid run for the update
all other payloads need to start at or after the first valid run for the update.
The globaltag needs to be gap and overlap free
All payloads in the staging tag should have as last iov an open iov (i.e. valid to infinity) but this can be disabled.
The script will check all the above requirements and will then calculate the necessary operations to
Add all payloads from the staging tag where a start validity of (0, 0) is replaced by the starting run for which this update should be valid.
close all iovs for payloads in the running tags just before the corresponding iov of the same payload in the staging tag, so either at the first run for the update to be valid or later
Optionally, make sure all payloads in the staging tag end in an open iov.
Example:
running tag contains
payload1, rev 1, valid from 0,1 to 1,0 payload1, rev 2, valid from 1,1 to -1,-1 payload2, rev 1, valid from 0,1 to -1,-1 payload3, rev 1, valid from 0,1 to 1,0 payload4, rev 1, valid from 0,1 to -1,-1 payload5, rev 1, valid from 0,1 to -1,-1staging tag contains
payload1, rev 3, valid from 0,0 to 1,8 payload1, rev 4, valid from 1,9 to 1,20 payload2, rev 2, valid from 1,5 to 1,20 payload3, rev 2, valid from 0,0 to -1,-1 payload4, rev 1, valid from 0,0 to 1,20Then running
b2conditionsdb tag runningupdate running staging --run 1 2 --allow-closed
, the running globaltag after the update will containpayload1, rev 1, valid from 0,1 to 1,0 payload1, rev 2, valid from 1,1 to 1,1 payload1, rev 3, valid from 1,2 to 1,8 payload1, rev 4, valid from 1,9 to 1,20 payload2, rev 1, valid from 0,1 to 1,4 payload2, rev 2, valid from 1,5 to 1,20 payload3, rev 1, valid from 0,1 to 1,0 payload3, rev 2, valid from 1,2 to -1,-1 payload4, rev 1, valid from 0,1 to 1,20 payload5, rev 1, valid from 0,1 to -1,-1Note that
the start of payload1 and payload3 in staging has been adjusted
payload2 in the running tag as been closed at 1,4, just before the validity from the staging tag
payload3 was already closed in the running tag so no change is performed. This might result in gaps but is intentional
payload4 was not closed at rim 1,2 because the staging tag had the same revision of the payload so the these were merged to one long validity.
payload5 was not closed as there was no update to it in the staging tag. If we would have run with
--full-replacement
it would have been closed.if we would have chosen
--run 1 1
the update would have failed because payload1, rev2 in running starts at 1,1 so we would have a conflictif we would have chosen
--run 1 6
the update would have failed because payload2 in the staging tag starts before this runif we would have chosen to open the final iovs in staging by using
--fix-closed
, payload1, rev 4; payload2, rev 2 and payload4 rev 1 would be valid until -1,-1 after the running tag. In fact, payload 4 would not be changed at all.
b2conditionsdb tag runningupdate [-h] [--debugging] [--help-full]
[--base-url BASE_URL]
[--auth-token AUTH_TOKEN] -r EXP RUN
[--allow-closed | --fix-closed | --simple-mode | --full-replacement]
[--dry-run]
running staging
Required Arguments
- running
name of the running globaltag
- staging
name of the staging globaltag
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).- --allow-closed
if given allow payloads in the staging tag to not be open, i.e. they don’t have to be open ended in the update. Useful to retire a payload by adding one last update
- --fix-closed
if given automatically open the last iov for each payload in staging if it is closed.
- --simple-mode
if given require the staging tag to solely consist of fully infinite validities: Only one iov per payload with a validity of (0,0,-1,-1)
- --full-replacement
if given perform a full replacement and close all open iovs in the running tag not present in the staging tag. After such an update exactly the payloads in the staging tag will be valid after the given run. This allows for closed iovs in the staging tag as with
--allow-closed
- --dry-run
Only show the changes, don’t try to apply them
required named arguments
- -r, --run
First experiment + run number for which the update should be valid. Two numbers separated by space
Sub-command b2conditionsdb tag show
#
This command will show details for the given globaltags like name, description and number of payloads.
b2conditionsdb tag show [-h] [--debugging] [--help-full] [--base-url BASE_URL]
[--auth-token AUTH_TOKEN]
TAGNAME [TAGNAME ...]
Required Arguments
- TAGNAME
globaltags to show
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).
Sub-command b2conditionsdb tag state
#
This command changes the state of a globaltag to the given value.
Usually the valid states are
- OPEN
Tag can be modified, payloads and iovs can be created and deleted. This is the default state for new or cloned globaltags and is not suitable for use in data analysis
Can be transitioned to TESTING, RUNNING
- TESTING
Tag cannot be modified and is suitable for testing but can be reopened
Can be transitioned to VALIDATED, OPEN
- VALIDATED
Tag cannot be modified and has been tested.
Can be transitioned to PUBLISHED, OPEN
- PUBLISHED
Tag cannot be modified and is suitable for user analysis
Can only be transitioned to INVALID
- RUNNING
Tag can only be modified by adding new runs, not modifying the payloads for existing runs.
- INVALID:
Tag is invalid and should not be used for anything.
This state is end of life for a globaltag and cannot be transitioned to any other state.
New in version release-04-00-00.
b2conditionsdb tag state [-h] [--debugging] [--help-full]
[--base-url BASE_URL] [--auth-token AUTH_TOKEN]
TAGNAME STATE
Required Arguments
- TAGNAME
globaltag to be changed
- STATE
new state for the globaltag
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).
4.6.8. Sub-command b2conditionsdb upload
#
This command allows uploading a local database which was created by basf2 to the central database. It assumes that the globaltag already exists so please create it before if necessary using ‘tag create’.
The command requires the tagname to upload the payloads to and a database.txt containing the payloads and their iovs. One can supply a directory where to look for the payloads, by default they are assumed to be in the same directory as the database text file.
b2conditionsdb upload [-h] [--debugging] [--help-full] [--base-url BASE_URL]
[--auth-token AUTH_TOKEN]
[--normalize | --normalize-name NORMALIZE_NAME]
[-j NPROCESS] [--retries RETRIES] [--ignore-existing]
TAGNAME PAYLOADSFILE
Required Arguments
- TAGNAME
globaltag to use for iov creation
- PAYLOADSFILE
Testing payload storage file containing list of iovs
Optional Arguments
- --debugging
Enable debugging of http traffic
- --base-url
URI for the base of the REST API, if not given a list of default locations is tried
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable
$BELLE2_CDB_AUTH_TOKEN
points to a file with a valid token, such token is used (useful for automatic workflows).- --normalize
Normalize the payload files to have reproducible checksums. This option should only be used if the payload files were created with an older software version (before release-04)
- --normalize-name
Set the file name in the root file metadata to the given value. This implicitly enables
--normalize
and should only be used if the payload files were created with an older software version (before release-04)- -j
Number of concurrent connections to use for database connection (default: 1)
- --retries
Number of retries on connection problems (default: 3)
- --ignore-existing
Don’t check if payloads or iovs already exist in database. Speeds up initialization as the list of existing payloads doesn’t need to be downloaded. Can be used on first upload but the script cannot resume an upload if this option is given
4.6.9. b2conditionsdb-extract
: Extract payload for a run range and save as TTree#
New in version release-03-00-00.
This tool reads the given conditions objects from the database and writes them to a TTree. Intra run dependencies are not supported.
For example the command
b2conditionsdb-extract --exp 5 --runs 1-3,12 BeamParameters
will create a root file with a Tree containing the beam parameters of the runs 1, 2, 3, and 12 of experiment 5. The branch BeamParameters_valid contains 1 if the beam parameters are available for the given run and 0 if not. The following command plots the x position of the beam spot after opening the conditions.root file in root:
conditions->Draw("BeamParameters.m_vertex.X():run", "BeamParameters_valid", "*")
usage: b2conditionsdb-extract [-h] [--output OUTPUT] [--tag TAG] --exp EXP
--runs RUNS
condition [condition ...]
Required Arguments
- condition
payload identifier, if the type is not equal to the name it has to be appended after a /
Optional Arguments
- --output, -o
output file containing the TTree of conditions
Default: “conditions.root”
- --tag, -t
global tag
- --exp, -e
experiment number
- --runs
run numbers, ranges are given as X-Y, multiple runs or ranges are separated by comma
4.6.10. b2conditionsdb-recommend
: Recommend a global tag to analyse a given file#
New in version release-03-00-00.
This tool determines the recommended set of global tags.
usage: b2conditionsdb-recommend [-h] [--oneline] [input_file]
Required Arguments
- input_file
Input file that should be processed. No input file means MC generation.
Optional Arguments
- --oneline
Prints the recommended global tags as a one-liner string.
4.6.11. b2conditionsdb-request
: Request inclusion of payloads in official global tag#
New in version release-03-00-00.
Tool to request an update of a global tag.
The tool will ask for the information required by the global tag manager for the update or take them from the command line options.
usage: b2conditionsdb-request [-h]
[--task {main,new,modify,validation,online,prompt,data,mc,analysis}]
[--normalize | --normalize-name NORMALIZE_NAME]
[--release RELEASE] [--reason REASON]
[--details DETAILS | --details-file DETAILS_FILE]
[--issue ISSUE] [--password PASSWORD | --oauth]
[--confirm] [--auth-token AUTH_TOKEN]
payloads
Required Arguments
- payloads
Name of the testing payload storage file
Optional Arguments
- --task
Possible choices: main, new, modify, validation, online, prompt, data, mc, analysis
- Type of global tag that should be updated.
Use new/modify if the request contains new/modified payload classes.
- --normalize
Normalize the payload files to have reproducible checksums. This option should only be used if the payload files were created with an older software version (before release-04)
- --normalize-name
Set the file name in the root file metadata to the given value. This implicitly enables
--normalize
and should only be used if the payload files were created with an older software version (before release-04)- --release
Required release version
- --reason
Reason for the request
- --details
Detailed instructions/information for the global tag manager
- --details-file
File containing detailed instructions/information for the global tag manager
- --issue
Identifier of an existing jira issue for the request
- --password
Password to authenticate the request
- --oauth
Authenticate via oauth. This is should only be used for automated updates running within the DAQ network.
- --confirm
Confirm that the given information is correct and that the payloads should be uploaded
- --auth-token
JSON Web Token necessary for authenticating to the conditions database. Useful only for debugging, since by the default the tool automatically gets a token for you by asking the B2MMS username and password. If the environment variable $BELLE2_CDB_AUTH_TOKEN points to a file with a valid token, such token is used (useful for automatic workflows).