Belle II Software light-2509-fornax
MetavariableDataTypeTest Class Reference
Inheritance diagram for MetavariableDataTypeTest:
Collaboration diagram for MetavariableDataTypeTest:

Public Member Functions

int process_file (self, str filepath)
 
 test_metavariable_data_types (self)
 

Static Public Attributes

dict hardcoded
 we have to hardcode some special cases as data types can't be obtained from source code directly
 
 registering_regex
 regex for finding the REGISTER_METAVARIABLE statements.
 
 extract_regex
 regex for extracting the function name and the enum type from REGISTER_METAVARIABLE statements # noqa
 
 lambda_type_regex = re.compile(r"-> (?P<lambdatype>double|bool|int)")
 regex for extracting the type of the lambda function in metavariable function definition # noqa
 

Detailed Description

Determine metavariable types from source code and assert correctness

Definition at line 41 of file test_variables_meta.py.

Member Function Documentation

◆ process_file()

int process_file ( self,
str filepath )
Check all metavariable types for specified file.

Args:
    filepath (str): path to file containing REGISTER_METAVARIABLE

Raises:
    AssertionError: Raised if no expected function definition is found.
    AssertionError: Rased if lambda function has no associated
                    type information, or no lambda function is defined.

Returns:
    int: number of metavariables in file.
    Used for sanity checks of the coverage.

Definition at line 70 of file test_variables_meta.py.

70 def process_file(self, filepath: str) -> int:
71 """Check all metavariable types for specified file.
72
73 Args:
74 filepath (str): path to file containing REGISTER_METAVARIABLE
75
76 Raises:
77 AssertionError: Raised if no expected function definition is found.
78 AssertionError: Rased if lambda function has no associated
79 type information, or no lambda function is defined.
80
81 Returns:
82 int: number of metavariables in file.
83 Used for sanity checks of the coverage.
84 """
85
86 # Read file contents
87 with open(filepath) as fp:
88 filecontent = fp.read()
89
90 # List for all found registering statements
91 registering_statements = self.registering_regex.findall(filecontent)
92 # Preprocess all statements by removing spaces and newlines
93 # This makes extraction of function name and enum type easier
94 registering_statements = list(
95 map(
96 lambda line: line.replace(" ", "").replace("\n", ""),
97 registering_statements,
98 ),
99 )
100
101 for statement in registering_statements:
102 # Extract enum type and name
103 match_content = self.extract_regex.match(statement)
104 function_name = match_content.groupdict()["function_name"]
105 enumtype = match_content.groupdict()["enumtype"]
106
107 if function_name in self.hardcoded:
108 self.assertEqual(
109 enumtype,
110 self.hardcoded[function_name],
111 f"Metavariable '{function_name}' in file {filepath}:\n"
112 f"Metavariable function return type and Manager::VariableDataType have to match.\n" # noqa
113 f"Expected: Manager::VariableDataType::{self.hardcoded[function_name]}, actual: Manager::VariableDataType::{enumtype}", # noqa
114 )
115 continue
116
117 # compile regex for finding metavariable function definition
118 function_definition_regex = re.compile(
119 r"Manager::FunctionPtr %s\‍(.*\‍)[^\{]*" % function_name
120 )
121 # compile regex for regular typed function definition
122 regular_definition_regex = re.compile(
123 r"(?P<return_type>double|int|bool) %s\‍(.*?\‍)" % function_name
124 )
125
126 # Find function body start with regex
127 definition = function_definition_regex.search(filecontent)
128 if definition is not None:
129 func_body_start = definition.end()
130 else: # Manager::FunctionPtr definition not found
131 # search for a regular typed function
132 return_type = (
133 regular_definition_regex.search(filecontent)
134 .groupdict()
135 .get("return_type")
136 ) # noqa
137 if return_type is not None:
138 self.assertEqual(
139 return_type,
140 enumtype,
141 f"Metavariable '{function_name}' in file {filepath}:\n"
142 "Metavariable function return type and Manager::VariableDataType have to match." # noqa
143 f"Return type is {return_type} but it is registered as Manager::VariableDataType::c_{enumtype}.\n", # noqa
144 )
145 continue
146 else:
147 raise AssertionError(
148 f"Metavariable '{function_name}' in file {filepath}:\n"
149 "Metavariable function return type and Manager::VariableDataType have to match." # noqa
150 "Return type of function could not be automatically determined from the source code." # noqa
151 "You can add an exception by adding the expected return type information to " # noqa
152 "the 'hardcoded' dictionary of this testcase."
153 )
154
155 # grab function body end
156 func_body_end = func_body_start + findMatchedParenthesis(
157 filecontent[func_body_start:], "{", "}"
158 )
159 # Get function body slice from filecontent
160 func_body = filecontent[func_body_start:func_body_end]
161
162 # grab lambda type definition
163 lambdatype_match = self.lambda_type_regex.search(func_body)
164 if lambdatype_match is not None:
165 lambdatype = lambdatype_match.groupdict()["lambdatype"]
166 self.assertEqual(
167 lambdatype,
168 enumtype,
169 f"Metavariable '{function_name}' in file {filepath}:\n"
170 f"Lambda function has return type {lambdatype} "
171 f"but is registered with Manager::VariableDataType::c_{enumtype}.\n" # noqa
172 "VariableDataType and lambda return type have to match.",
173 )
174 else: # lambda type or definition not found
175 raise AssertionError(
176 f"Metavariable '{function_name}' in {filepath}:\n"
177 "VariableDataType and lambda definition have to match.\n"
178 "Either lambda function is missing return type information"
179 ", or lambda definition could not be found.\n" # noqa
180 "Please add return type annotation '(const Particle * particle) -> double/int/bool' to lambda.\n" # noqa
181 "Or add this metavariable as exception, by adding the expected return type information in the 'hardcoded' dictionary of this testcase\n" # noqa
182 f"{func_body}"
183 )
184
185 # Return the number of registering statements in this file
186 return len(registering_statements)
187

◆ test_metavariable_data_types()

test_metavariable_data_types ( self)
Metavariables have to be registered with the correct Manager::Variable::VariableDataType enum value. This test makes sure Metavariable definition and variable registration are correct.

Definition at line 188 of file test_variables_meta.py.

188 def test_metavariable_data_types(self):
189 """Metavariables have to be registered with the correct Manager::Variable::VariableDataType enum value. This test makes sure Metavariable definition and variable registration are correct.""" # noqa
190 # check if grep is available
191 try:
192 subprocess.run(["grep", "-V"], check=True, capture_output=True)
193 except subprocess.CalledProcessError:
194 logging.basicConfig(format="%(message)s")
195 logging.error(
196 "TEST SKIPPED: MetavariableDataTypeTest skipped because grep is not available." # noqa
197 )
198 self.fail()
199
200 # Use grep to find files with REGISTER_METAVARIABLE statements
201 analysis_module = basf2.find_file("analysis")
202 files = subprocess.run(
203 [
204 "grep",
205 "REGISTER_METAVARIABLE",
206 "-r",
207 analysis_module,
208 "-I",
209 "-l",
210 ],
211 capture_output=True,
212 )
213 # Decode stdout and extract filenames
214 files = files.stdout.decode().split("\n")
215 files = list(filter(lambda file: file.endswith(".cc"), files))
216
217 num_files = len(files)
218 print(f"Number of files including meta-variables is {num_files}")
219
220 # There should be at least 15 files
221 self.assertGreaterEqual(num_files, 15)
222 # We track the number of metavariables to make sure we don't miss some
223 num_metavariables = 0
224 for filepath in files:
225 num_metavariables += self.process_file(filepath)
226
227 # We should get at least 243 registering statements
228 print(f"Number of meta-variables is {num_metavariables}")
229 self.assertGreaterEqual(num_metavariables, 238)
230
231

Member Data Documentation

◆ extract_regex

extract_regex
static
Initial value:
= re.compile(
r'REGISTER_METAVARIABLE\‍(".*?",(?P<function_name>[^,]*),.*?Manager::VariableDataType::c_(?P<enumtype>double|bool|int)' # noqa
)

regex for extracting the function name and the enum type from REGISTER_METAVARIABLE statements # noqa

Definition at line 64 of file test_variables_meta.py.

◆ hardcoded

dict hardcoded
static
Initial value:
= {
"softwareTriggerResult": "double",
"formula": "double",
"softwareTriggerResultNonPrescaled": "double",
"isDaughterOfList": "bool",
"isGrandDaughterOfList": "bool",
"daughterDiffOf": "double",
"grandDaughterDiffOf": 'double',
}

we have to hardcode some special cases as data types can't be obtained from source code directly

Definition at line 48 of file test_variables_meta.py.

◆ lambda_type_regex

lambda_type_regex = re.compile(r"-> (?P<lambdatype>double|bool|int)")
static

regex for extracting the type of the lambda function in metavariable function definition # noqa

Definition at line 68 of file test_variables_meta.py.

◆ registering_regex

registering_regex
static
Initial value:
= re.compile(
r"(?s)REGISTER_METAVARIABLE.*?Manager::VariableDataType::(?:c_double|c_int|c_bool)" # noqa
)

regex for finding the REGISTER_METAVARIABLE statements.

Definition at line 60 of file test_variables_meta.py.


The documentation for this class was generated from the following file: