TestingPayloadStorage Class Reference

Class to store and retrieve temporary payloads. More...

#include <TestingPayloadStorage.h>

Collaboration diagram for TestingPayloadStorage:

Public Member Functions
	TestingPayloadStorage (const std::string &filename)
	Create a new instance to work on a given filename.
bool	get (const EventMetaData &event, PayloadMetadata &info)
	Try to fill the PayloadMetaData for the given EventMetaData, return true on success, false if no machting payload could be found.
bool	storeData (const std::string &name, TObject *object, const IntervalOfValidity &iov)
	Store a TObject instance as a payload with given name and interval of validity. More...
bool	storePayload (const std::string &name, const std::string &fileName, const IntervalOfValidity &iov)
	Store an existing file as payload with given name and interval of validity. More...
void	reset ()
	Reset the list of known payloads. More...

Private Member Functions
void	read ()
	Read the given storage file, done lazily on first access to get() after construction or call to reset()
bool	writePayload (const std::string &fileName, const std::string &name, const TObject *object)
	Write a payload file from the given object and name. More...
bool	store (const std::string &name, const IntervalOfValidity &iov, const std::function< bool(const std::string &)> &writer)
	Try to store a new payload with the given name and interval of validity. More...

Static Private Member Functions
static std::string	payloadFilename (const std::string &path, const std::string &name, int revision)
	Build the filename for a new payload with a given name and revision in a directory.

Private Attributes
std::unordered_map< std::string, std::vector< std::tuple< size_t, IntervalOfValidity > > >	m_payloads
	Map of known payloads to a list of known revisions and their interval of validity.
std::string	m_filename
	Storage file where to look for payloads. More...
std::string	m_absoluteFilename
	Storage file where to look for payloads converted to an absolute path to be robust against directory changes.
std::string	m_payloadDir
	Directory containing the storage file as absolute file name. More...
bool	m_initialized {false}
	Remember whether we read the file already.

Detailed Description

Class to store and retrieve temporary payloads.

Temporary payloads are stored in a directory where we create a root files and a plain text file with name, revision and iov for each of these files. This is not very safe but ideal for testing and manual adjustments before final validation and upload to the cental server.

The interface is very basic: We create an instance with a filename and then we either get() payload information for a given event and payload metadata struct or we store a given payload either from an object (storeData()) or from an existing file (storePayload()).

Both call the underlying store() which tries to be robust against multiple processes trying to create payloads at the same time.

Definition at line 49 of file TestingPayloadStorage.h.

Member Function Documentation

reset()

void reset ( )

inline

Reset the list of known payloads.

Will trigger re-reading the file on next access to get()

Definition at line 65 of file TestingPayloadStorage.h.

store()

bool store ( const std::string &  name, const IntervalOfValidity &  iov, const std::function< bool(const std::string &)> &  writer  )

private

Try to store a new payload with the given name and interval of validity.

This function first tries to find the next free revision and then call the writer function to write the payload to the given filename. If the writer function returns success the payload is added to the storage file

Definition at line 166 of file TestingPayloadStorage.cc.

                                 {
      fs::create_directories(m_payloadDir);
    }
    // get lock for write access to database file
    FileSystem::Lock lock(m_absoluteFilename);
    if (!lock.lock()) {
      B2ERROR("Locking of testing payload storage file failed, cannot create payload"
              << LogVar("storage file", m_filename));
      return false;
    }
    std::ofstream file(m_absoluteFilename.c_str(), std::ios::app);
    if (!file.is_open()) {
      B2ERROR("Could not open testing payload storage file for writing" << LogVar("storage file", m_filename));
    }
 
    // Find the next free revision number
    for (int revision = 1; revision < INT_MAX; ++revision) {
      auto filename = payloadFilename(m_payloadDir, name, revision);
      // FIXME: This could be a race condition, we check for existence and then
      // create which could fail if two processes try this at the same time and
      // thus overwrite the files of each other. We could instead check if an
      // `open(filename.c_str(), O_CREAT|O_EXCL)` is successful in which the
      // file is ours however I'm a bit sceptical if this will work on SL6. But
      // since we locked the database file and no longer support the payloads be
      // in a different directory then the text files this almost fine. However
      // there could still be multiple text files in the same directory so still
      // a slight chance for race conditions. Or locking could just not work on
      // some file systems, for example misconfigured NFS
      if (FileSystem::fileExists(filename)) continue;
      // free revision found, try to save
      if (!writer(filename)) return false;
      // Ok, now we need to add it to the database file
      file << "dbstore/" << name << " " << revision << " " << iov << std::endl;
      B2DEBUG(32, "Storing testing payload" << LogVar("storage file", m_filename) << LogVar("name", name)
              << LogVar("local revision", revision) << LogVar("iov", iov));
      // And make sure we reread the file on next request to payloads
      m_initialized = false;
      return true;
    }
    B2ERROR("Could not find a suitable revision to create payload" << LogVar("storage file", m_filename) << LogVar("name", name));
    return false;
  }
 
  bool TestingPayloadStorage::writePayload(const std::string& fileName, const std::string& name, const TObject* object)
  {
    // Save the current gDirectory
    TDirectory::TContext saveDir;
    // And create a reproducible TFile: one that has the same checksum every time it's created as long as the content is the same
    std::unique_ptr<TFile> file{TFile::Open((fileName + "?reproducible=PayloadFile").c_str(), "RECREATE")};
    if (!file || !file->IsOpen()) {

storeData()

bool storeData	(	const std::string &	name,
		TObject *	object,
		const IntervalOfValidity &	iov
	)

Store a TObject instance as a payload with given name and interval of validity.

This will create a new ROOT file containing the object in the directory of the storage file

Definition at line 144 of file TestingPayloadStorage.cc.

storePayload()

bool storePayload	(	const std::string &	name,
		const std::string &	fileName,
		const IntervalOfValidity &	iov
	)

Store an existing file as payload with given name and interval of validity.

This will create a copy of the file with a different name in the directory of the storage file

Definition at line 151 of file TestingPayloadStorage.cc.

writePayload()

bool writePayload ( const std::string &  fileName, const std::string &  name, const TObject *  object  )

private

Write a payload file from the given object and name.

Will create new root file containing object under the Key name with the name fileName

Definition at line 220 of file TestingPayloadStorage.cc.

Member Data Documentation

m_filename

std::string m_filename

private

Storage file where to look for payloads.

This is the logical file name as given by the user

Definition at line 72 of file TestingPayloadStorage.h.

m_payloadDir

std::string m_payloadDir

private

Directory containing the storage file as absolute file name.

Will try to create it if it doesn't exist and storeData() or storePayload() is called

Definition at line 78 of file TestingPayloadStorage.h.

---

The documentation for this class was generated from the following files:

• framework/database/include/TestingPayloadStorage.h
• framework/database/src/TestingPayloadStorage.cc