Session Class Reference

A wrapper around Ort::Session providing model execution. More...

#include <ONNX.h>

Public Member Functions
	Session (const std::string filename)
	Constructs a new ONNX Runtime Session using the specified model file.
void	run (const std::map< std::string, std::shared_ptr< BaseTensor > > &inputMap, const std::map< std::string, std::shared_ptr< BaseTensor > > &outputMap)
	Runs inference on the model using named Tensor maps.
void	run (const std::vector< const char * > &inputNames, std::vector< Ort::Value > &inputs, const std::vector< const char * > &outputNames, std::vector< Ort::Value > &outputs)
	Runs inference on the model using raw ONNX Runtime inputs and outputs.

Private Attributes
Ort::Env	m_env
	Environment object for ONNX session.
Ort::SessionOptions	m_sessionOptions
	ONNX session configuration.
std::unique_ptr< Ort::Session >	m_session
	The ONNX inference session.
Ort::RunOptions	m_runOptions
	Options to be passed to Ort::Session::Run.

Detailed Description

A wrapper around Ort::Session providing model execution.

This class encapsulates an ONNX Runtime session, pre-configured with default settings such as single-threaded execution. It offers a more user-friendly interface to run inference using a custom Tensor class, which is easier to work with than raw Ort::Value instances.

Example usage:

#include <mva/methods/ONNX.h>
 
using Belle2::MVA::ONNX::Tensor;
 
// ...
 
// assume my_model.onnx contains a model with inputs
// - "a": float Tensor with shape (1, 3)
// - "b": int64 Tensor with shape (1, 8, 5)
// and one float output called "output" with shape (1, 5)
Belle2::MVA::ONNX::Session session("my_model.onnx");
 
auto input_a = Tensor<float>::make_shared(3, {1, 3});
auto input_b = Tensor<int64_t>::make_shared(8 * 5, {1, 8, 5});
auto output = Tensor<float>::make_shared(5, {1, 5});
 
// example for filling data using multi dimensional indexing
input_b->at({0, 2, 4}) = 42;
 
// run model and fill output values
session.run({{"a", input_a}, {"b", input_b}}, {{"output", output}});
 
// get 3rd output value - example for 1-dimensional indexing
int output_3 = output->at(3);

Note

This method will not work with Tensor<bool> since the underlying std::vector<bool> does not support getting a pointer to an array. If you have a model with boolean inputs, either convert it to accept a different type (e.g. uint8_t) or use the Session::run overload thet works directl with Ort::Value instances.

Definition at line 301 of file ONNX.h.

Constructor & Destructor Documentation

Session()

Session

(

const std::string

filename

)

Constructs a new ONNX Runtime Session using the specified model file.

Parameters

filename

Path to the ONNX model file.

Definition at line 18 of file ONNX.cc.

{
  // Ensure single-threaded execution, see
  // https://onnxruntime.ai/docs/performance/tune-performance/threading.html
  //
  // InterOpNumThreads is probably optional (not used in ORT_SEQUENTIAL mode)
  // Also, with batch size 1 and ORT_SEQUENTIAL mode, MLP-like models will
  // always run single threaded, but maybe not e.g. graph networks which can
  // run in parallel on nodes. Here, setting IntraOpNumThreads to 1 is
  // important to ensure single-threaded execution.
  m_sessionOptions.SetIntraOpNumThreads(1);
  m_sessionOptions.SetInterOpNumThreads(1);
  m_sessionOptions.SetExecutionMode(ORT_SEQUENTIAL); // default, but make it explicit
 
  m_session = std::make_unique<Ort::Session>(m_env, filename.c_str(), m_sessionOptions);
}

Member Function Documentation

run() [1/2]

void run	(	const std::map< std::string, std::shared_ptr< BaseTensor > > &	inputMap,
		const std::map< std::string, std::shared_ptr< BaseTensor > > &	outputMap )

Runs inference on the model using named Tensor maps.

This overload accepts maps of input and output tensor names to their corresponding tensor data. It feeds the input tensors into the session and fills the output tensors with inference results.

Parameters

inputMap	A map of input tensor names to shared pointers of Tensor objects.
outputMap	A map of output tensor names to shared pointers of Tensor objects.

Definition at line 35 of file ONNX.cc.

{
  std::vector<Ort::Value> inputs;
  std::vector<Ort::Value> outputs;
  std::vector<const char*> inputNames;
  std::vector<const char*> outputNames;
  for (auto& x : inputMap) {
    inputNames.push_back(x.first.c_str());
    inputs.push_back(std::move(x.second->createOrtTensor()));
  }
  for (auto& x : outputMap) {
    outputNames.push_back(x.first.c_str());
    outputs.push_back(std::move(x.second->createOrtTensor()));
  }
  run(inputNames, inputs, outputNames, outputs);
}

run() [2/2]

void run	(	const std::vector< const char * > &	inputNames,
		std::vector< Ort::Value > &	inputs,
		const std::vector< const char * > &	outputNames,
		std::vector< Ort::Value > &	outputs )

Runs inference on the model using raw ONNX Runtime inputs and outputs.

This overload provides works with raw ONNX Runtime Ort::Value objects and names. It's a lower level abstraction that should usually not be directly used, except in performance critical applications (meaning situations where saving O(1 microsecond) per call is helpful).

Parameters

inputNames	A vector of input tensor names.
inputs	A vector of Ort::Value objects representing the input tensors.
outputNames	A vector of output tensor names.
outputs	A vector of Ort::Value objects to be filled with the model's outputs.

Definition at line 53 of file ONNX.cc.

{
  m_session->Run(m_runOptions, inputNames.data(), inputs.data(), inputs.size(),
                 outputNames.data(), outputs.data(), outputs.size());
}

Member Data Documentation

m_env

Ort::Env m_env

private

Environment object for ONNX session.

Definition at line 345 of file ONNX.h.

m_runOptions

Ort::RunOptions m_runOptions

private

Options to be passed to Ort::Session::Run.

Definition at line 360 of file ONNX.h.

m_session

std::unique_ptr<Ort::Session> m_session

private

The ONNX inference session.

Definition at line 355 of file ONNX.h.

m_sessionOptions

Ort::SessionOptions m_sessionOptions

private

ONNX session configuration.

Definition at line 350 of file ONNX.h.

---

The documentation for this class was generated from the following files:

• mva/methods/include/ONNX.h
• mva/methods/src/ONNX.cc