LocalMetadataProvider Class Reference

Public Member Functions
def	__init__ (self, filename, mode="read")
def	get_payload_count (self)
def	add_globaltag (self, tag_id, name, state, iovs)
def	get_globaltags (self)
def	get_payloads (self)
def	get_all_iovs (self, globalTag, exp=None, run=None, message=None)

Static Public Attributes
int	APPLICATION_ID = 0xb2cdb
	Application ID to be stored int the sqlite file.
int	SCHEMA_VERSION = 1
	Schema version, to be increased when the table definitions change so that we can check for safe append operation.
str	SCHEMA_SQL
	SQL script to create all necessary tables and views.

Protected Member Functions
def	_resolve_id (self, name, entity)

Protected Attributes
	_cache
	Cache name->id mappings from the database.
	_database
	sqlite Database connection

Detailed Description

Class to handle local sqlite dump of conditions database metadata

This class can create and read sqlite dumps of the central database in a format
compatible with the local metadata provider in basf2.

Definition at line 20 of file local_metadata.py.

Constructor & Destructor Documentation

__init__()

def __init__	(		self,
			filename,
			mode = "read"
	)

Open an sqlite database and make sure that the schema exists in the
correct version or create it if ``mode=overwrite``

Arguments:
  filename (str): name of the database file
  readonly (str): how to open the file. Can be one of ``read`` to open
    the file readonly, ``append` to append new data to an existing file
    and ``overwrite`` to recreate all tables and overwrite the contents.

Definition at line 124 of file local_metadata.py.

    def __init__(self, filename, mode="read"):
        """
        Open an sqlite database and make sure that the schema exists in the
        correct version or create it if ``mode=overwrite``
 
        Arguments:
          filename (str): name of the database file
          readonly (str): how to open the file. Can be one of ``read`` to open
            the file readonly, ``append` to append new data to an existing file
            and ``overwrite`` to recreate all tables and overwrite the contents.
        """
 
        
        self._cache = {}
        
        self._database = None
 
        # connect to the database file ...
        if mode == "read":
            self._database = sqlite3.connect(f"file:{filename}?mode=ro", uri=True)
        elif mode in ["append", "overwrite"]:
            self._database = sqlite3.connect(filename)
        else:
            raise RuntimeError("invalid mode: please supply one of 'read', 'append', 'overwrite'")
 
        if mode == "overwrite":
            # drop and recreate all tables
            self._database.executescript(self.SCHEMA_SQL)
            # and set the application id/schema version correctly
            self._database.execute(f"PRAGMA application_id = {self.APPLICATION_ID}")
            self._database.execute(f"PRAGMA user_version = {self.SCHEMA_VERSION}")
            self._database.commit()
        else:
            # make sure the application id/schema version is the same
            application_id = self._database.execute("PRAGMA application_id").fetchone()[0]
            if application_id != self.APPLICATION_ID:
                raise RuntimeError("Not a b2conditionsdb database file")
            schema_version = self._database.execute("PRAGMA user_version").fetchone()[0]
            if schema_version != self.SCHEMA_VERSION:
                raise RuntimeError("Cannot use sqlite file: different schema version, please recreate")
 

Member Function Documentation

_resolve_id()

def _resolve_id (   self,   name,   entity  )

protected

Resolve the id for a named entity in the database file.

Create new entities on demand and cache all known entities

Parameters:
  name (str): name to lookup
  entity (str): type of the entity, currently ``baseUrl`` or ``payloadName``

Definition at line 170 of file local_metadata.py.

    def _resolve_id(self, name, entity):
        """
        Resolve the id for a named entity in the database file.
 
        Create new entities on demand and cache all known entities
 
        Parameters:
          name (str): name to lookup
          entity (str): type of the entity, currently ``baseUrl`` or ``payloadName``
        """
        # fill existing entries on first access
        if entity not in self._cache:
            self._cache[entity] = {row[0]: row[1] for row in self._database.execute(f"SELECT {entity}, {entity}Id from {entity}s")}
        # and then check if we have this entry, otherwise make a new one
        cache = self._cache[entity]
        if name not in cache:
            cache[name] = len(cache) + 1
            self._database.execute(f"INSERT INTO {entity}s ({entity}Id, {entity}) VALUES (?,?)", (cache[name], name))
        return cache[name]
 

add_globaltag()

def add_globaltag	(		self,
			tag_id,
			name,
			state,
			iovs
	)

Add a globaltag to the database file. If the globaltag already exists in
the file its contents will be replaced.

Parameters: tag_id (str): id of the globaltag in the central database
  name (str): name of the globaltag in the central database state (str):
  state of the globaltag in the central database iovs
  (list(PayloadInformation)): all iovs valid for this globaltag

Definition at line 190 of file local_metadata.py.

    def add_globaltag(self, tag_id, name, state, iovs):
        """
        Add a globaltag to the database file. If the globaltag already exists in
        the file its contents will be replaced.
 
        Parameters: tag_id (str): id of the globaltag in the central database
          name (str): name of the globaltag in the central database state (str):
          state of the globaltag in the central database iovs
          (list(PayloadInformation)): all iovs valid for this globaltag
        """
        self._database.execute("INSERT OR REPLACE INTO globaltags VALUES (?,?,?)", (tag_id, name, state))
        # remove existing iovs ... we want to append globaltags but we want all globaltags to be correct
        self._database.execute("DELETE from iovs WHERE globalTagId=?", (tag_id,))
 
        all_payloads = {}
        all_iovs = []
        for p in iovs:
            if p.payload_id not in all_payloads:
                base_url = self._resolve_id(p.base_url, 'baseUrl')
                name = self._resolve_id(p.name, 'payloadName')
                url = None
                if p.payload_url.lstrip('/') != f"dbstore/{p.name}/dbstore_{p.name}_rev_{p.revision}.root":
                    url = p.payload_url
                all_payloads[p.payload_id] = (p.payload_id, name, p.revision, p.checksum, url, base_url)
            all_iovs.append((tag_id, p.payload_id) + p.iov)
 
        self._database.executemany("INSERT OR REPLACE INTO payloads VALUES (?,?,?,?,?,?)", all_payloads.values())
        self._database.executemany("INSERT INTO iovs VALUES (?,?,?,?,?,?)", all_iovs)
        # make sure everything is committed
        self._database.commit()
        self._database.execute("VACUUM")
 

get_all_iovs()

def get_all_iovs	(		self,
			globalTag,
			exp = None,
			run = None,
			message = None
	)

Get all iovs for a given globaltag

Parameters:
  globalTag (str): name of the globaltag
  exp (int): experiment number to check (or None to return all iovs)
  run (int): run number to check (or None to return all iovs)
  message (str): ignored, just for compatibility with `ConditionsDB.get_all_iovs`

Returns:
  a sorted list of `PayloadInformation` objects

Definition at line 242 of file local_metadata.py.

    def get_all_iovs(self, globalTag, exp=None, run=None, message=None):
        """Get all iovs for a given globaltag
 
        Parameters:
          globalTag (str): name of the globaltag
          exp (int): experiment number to check (or None to return all iovs)
          run (int): run number to check (or None to return all iovs)
          message (str): ignored, just for compatibility with `ConditionsDB.get_all_iovs`
 
        Returns:
          a sorted list of `PayloadInformation` objects
        """
        params = {"globalTag": globalTag}
        query = """\
            SELECT
              payloadId, payloadName, revision, checksum, payloadUrl, baseUrl,
              firstExp, firstRun, finalExp, finalRun
            FROM iov_payloads
            WHERE globalTagName=:globalTag"""
        if exp is not None:
            params.update({"exp": exp, "run": run})
            query += """ AND\
                ((firstExp==:exp AND firstRun<=:run) OR firstExp<:exp) AND
                (finalExp<0 OR (finalRun<0 AND finalExp>=:exp) OR (finalExp>:exp) OR (finalExp==:exp AND finalRun>=:run))"""
        iovs = sorted([PayloadInformation(*row[:6], iov_id=None, iov=row[6:]) for row in
                       self._database.execute(query, params)])
        return iovs

get_globaltags()

def get_globaltags

(

self

)

Return the list of globaltags stored in the file

Returns:
  a list of (id, name, state) tuples for all globaltags

Definition at line 222 of file local_metadata.py.

    def get_globaltags(self):
        """Return the list of globaltags stored in the file
 
        Returns:
          a list of (id, name, state) tuples for all globaltags
        """
        return [row for row in self._database.execute("SELECT globalTagId, globalTagName, globalTagStatus FROM globaltags "
                                                      "ORDER by globalTagName")]
 

get_payload_count()

def get_payload_count

(

self

)

Get the number of distinct payloads known to this file

Definition at line 165 of file local_metadata.py.

    def get_payload_count(self):
        """Get the number of distinct payloads known to this file"""
        cursor = self._database.execute("SELECT count(*) from full_payloads")
        return cursor.fetchone()[0]
 

get_payloads()

def get_payloads

(

self

)

Get all payloads existing in this file

Returns:
  a sorted list of `PayloadInformation` objects for all payloads defined in this file with the iov set to None

Definition at line 231 of file local_metadata.py.

    def get_payloads(self):
        """Get all payloads existing in this file
 
        Returns:
          a sorted list of `PayloadInformation` objects for all payloads defined in this file with the iov set to None
        """
        payloads = sorted([PayloadInformation(*row) for row in
                           self._database.execute("SELECT payloadId, payloadName, revision, checksum, "
                                                  "payloadUrl, baseUrl from full_payloads")])
        return payloads
 

Member Data Documentation

_cache

_cache

protected

Cache name->id mappings from the database.

Definition at line 137 of file local_metadata.py.

_database

_database

protected

sqlite Database connection

Definition at line 139 of file local_metadata.py.

APPLICATION_ID

int APPLICATION_ID = 0xb2cdb

static

Application ID to be stored int the sqlite file.

Definition at line 28 of file local_metadata.py.

SCHEMA_SQL

str SCHEMA_SQL

static

SQL script to create all necessary tables and views.

Definition at line 33 of file local_metadata.py.

SCHEMA_VERSION

int SCHEMA_VERSION = 1

static

Schema version, to be increased when the table definitions change so that we can check for safe append operation.

Definition at line 31 of file local_metadata.py.

---

The documentation for this class was generated from the following file:

• framework/scripts/conditions_db/local_metadata.py