Belle II Software light-2406-ragdoll
MetavariableDataTypeTest Class Reference
Inheritance diagram for MetavariableDataTypeTest:
Collaboration diagram for MetavariableDataTypeTest:

Public Member Functions

int process_file (self, str filepath)
 
def test_metavariable_data_types (self)
 

Static Public Attributes

dict hardcoded
 we have to hardcode some special cases as data types can't be obtained from source code directly
 
re registering_regex
 regex for finding the REGISTER_METAVARIABLE statements.
 
re extract_regex
 regex for extracting the function name and the enum type from REGISTER_METAVARIABLE statements # noqa
 
re lambda_type_regex = re.compile(r"-> (?P<lambdatype>double|bool|int)")
 regex for extracting the type of the lambda function in metavariable function definition # noqa
 

Detailed Description

Determine metavariable types from source code and assert correctness

Definition at line 41 of file test_variables_meta.py.

Member Function Documentation

◆ process_file()

int process_file (   self,
str  filepath 
)
Check all metavariable types for specified file.

Args:
    filepath (str): path to file containing REGISTER_METAVARIABLE

Raises:
    AssertionError: Raised if no expected function definition is found.
    AssertionError: Rased if lambda function has no associated
                    type information, or no lambda function is defined.

Returns:
    int: number of metavariables in file.
    Used for sanity checks of the coverage.

Definition at line 77 of file test_variables_meta.py.

77 def process_file(self, filepath: str) -> int:
78 """Check all metavariable types for specified file.
79
80 Args:
81 filepath (str): path to file containing REGISTER_METAVARIABLE
82
83 Raises:
84 AssertionError: Raised if no expected function definition is found.
85 AssertionError: Rased if lambda function has no associated
86 type information, or no lambda function is defined.
87
88 Returns:
89 int: number of metavariables in file.
90 Used for sanity checks of the coverage.
91 """
92
93 # Read file contents
94 with open(filepath) as fp:
95 filecontent = fp.read()
96
97 # List for all found registering statements
98 registering_statements = self.registering_regex.findall(filecontent)
99 # Preprocess all statements by removing spaces and newlines
100 # This makes extraction of function name and enum type easier
101 registering_statements = list(
102 map(
103 lambda line: line.replace(" ", "").replace("\n", ""),
104 registering_statements,
105 ),
106 )
107
108 for statement in registering_statements:
109 # Extract enum type and name
110 match_content = self.extract_regex.match(statement)
111 function_name = match_content.groupdict()["function_name"]
112 enumtype = match_content.groupdict()["enumtype"]
113
114 if function_name in self.hardcoded:
115 self.assertEqual(
116 enumtype,
117 self.hardcoded[function_name],
118 f"Metavariable '{function_name}' in file {filepath}:\n"
119 f"Metavariable function return type and Manager::VariableDataType have to match.\n" # noqa
120 f"Expected: Manager::VariableDataType::{self.hardcoded[function_name]}, actual: Manager::VariableDataType::{enumtype}", # noqa
121 )
122 continue
123
124 # compile regex for finding metavariable function definition
125 function_definition_regex = re.compile(
126 r"Manager::FunctionPtr %s\‍(.*\‍)[^\{]*" % function_name
127 )
128 # compile regex for regular typed function definition
129 regular_definition_regex = re.compile(
130 r"(?P<return_type>double|int|bool) %s\‍(.*?\‍)" % function_name
131 )
132
133 # Find function body start with regex
134 definition = function_definition_regex.search(filecontent)
135 if definition is not None:
136 func_body_start = definition.end()
137 else: # Manager::FunctionPtr definition not found
138 # search for a regular typed function
139 return_type = (
140 regular_definition_regex.search(filecontent)
141 .groupdict()
142 .get("return_type")
143 ) # noqa
144 if return_type is not None:
145 self.assertEqual(
146 return_type,
147 enumtype,
148 f"Metavariable '{function_name}' in file {filepath}:\n"
149 "Metavariable function return type and Manager::VariableDataType have to match." # noqa
150 f"Return type is {return_type} but it is registered as Manager::VariableDataType::c_{enumtype}.\n", # noqa
151 )
152 continue
153 else:
154 raise AssertionError(
155 f"Metavariable '{function_name}' in file {filepath}:\n"
156 "Metavariable function return type and Manager::VariableDataType have to match." # noqa
157 "Return type of function could not be automatically determined from the source code." # noqa
158 "You can add an exception by adding the expected return type information to " # noqa
159 "the 'hardcoded' dictionary of this testcase."
160 )
161
162 # grab function body end
163 func_body_end = func_body_start + findMatchedParenthesis(
164 filecontent[func_body_start:], "{", "}"
165 )
166 # Get function body slice from filecontent
167 func_body = filecontent[func_body_start:func_body_end]
168
169 # grab lambda type definition
170 lambdatype_match = self.lambda_type_regex.search(func_body)
171 if lambdatype_match is not None:
172 lambdatype = lambdatype_match.groupdict()["lambdatype"]
173 self.assertEqual(
174 lambdatype,
175 enumtype,
176 f"Metavariable '{function_name}' in file {filepath}:\n"
177 f"Lambda function has return type {lambdatype} "
178 f"but is registered with Manager::VariableDataType::c_{enumtype}.\n" # noqa
179 "VariableDataType and lambda return type have to match.",
180 )
181 else: # lambda type or definition not found
182 raise AssertionError(
183 f"Metavariable '{function_name}' in {filepath}:\n"
184 "VariableDataType and lambda definition have to match.\n"
185 "Either lambda function is missing return type information"
186 ", or lambda definition could not be found.\n" # noqa
187 "Please add return type annotation '(const Particle * particle) -> double/int/bool' to lambda.\n" # noqa
188 "Or add this metavariable as exception, by adding the expected return type information in the 'hardcoded' dictionary of this testcase\n" # noqa
189 f"{func_body}"
190 )
191
192 # Return the number of registering statements in this file
193 return len(registering_statements)
194

◆ test_metavariable_data_types()

def test_metavariable_data_types (   self)
Metavariables have to be registered with the correct Manager::Variable::VariableDataType enum value. This test makes sure Metavariable definition and variable registration are correct.

Definition at line 195 of file test_variables_meta.py.

195 def test_metavariable_data_types(self):
196 """Metavariables have to be registered with the correct Manager::Variable::VariableDataType enum value. This test makes sure Metavariable definition and variable registration are correct.""" # noqa
197 # check if grep is available
198 try:
199 subprocess.run(["grep", "-V"], check=True, capture_output=True)
200 except subprocess.CalledProcessError:
201 logging.basicConfig(format="%(message)s")
202 logging.error(
203 "TEST SKIPPED: MetavariableDataTypeTest skipped because grep is not available." # noqa
204 )
205 self.fail()
206
207 # Use grep to find files with REGISTER_METAVARIABLE statements
208 analysis_module = basf2.find_file("analysis")
209 files = subprocess.run(
210 [
211 "grep",
212 "REGISTER_METAVARIABLE",
213 "-r",
214 analysis_module,
215 "-I",
216 "-l",
217 ],
218 capture_output=True,
219 )
220 # Decode stdout and extract filenames
221 files = files.stdout.decode().split("\n")
222 files = list(filter(lambda file: file.endswith(".cc"), files))
223
224 num_files = len(files)
225 print(f"Number of files including meta-variables is {num_files}")
226
227 # There should be at least 13 files
228 self.assertGreaterEqual(num_files, 13)
229 # We track the number of metavariables to make sure we don't miss some
230 num_metavariables = 0
231 for filepath in files:
232 num_metavariables += self.process_file(filepath)
233
234 # We should get at least 237 registering statements
235 print(f"Number of meta-variables is {num_metavariables}")
236 self.assertGreaterEqual(num_metavariables, 237)
237
238

Member Data Documentation

◆ extract_regex

re extract_regex
static
Initial value:
= re.compile(
r'REGISTER_METAVARIABLE\‍(".*?",(?P<function_name>[^,]*),.*?Manager::VariableDataType::c_(?P<enumtype>double|bool|int)' # noqa
)

regex for extracting the function name and the enum type from REGISTER_METAVARIABLE statements # noqa

Definition at line 71 of file test_variables_meta.py.

◆ hardcoded

dict hardcoded
static
Initial value:
= {
"softwareTriggerResult": "double",
"formula": "double",
"softwareTriggerResultNonPrescaled": "double",
"isDaughterOfList": "bool",
"isGrandDaughterOfList": "bool",
"daughterDiffOf": "double",
"daughterDiffOfPhi": "double",
"daughterDiffOfClusterPhi": "double",
"mcDaughterDiffOfPhi": "double",
"grandDaughterDiffOf": 'double',
"grandDaughterDiffOfPhi": "double",
"grandDaughterDiffOfClusterPhi": "double",
"daughterDiffOfPhiCMS": "double",
"daughterDiffOfClusterPhiCMS": "double",
}

we have to hardcode some special cases as data types can't be obtained from source code directly

Definition at line 48 of file test_variables_meta.py.

◆ lambda_type_regex

re lambda_type_regex = re.compile(r"-> (?P<lambdatype>double|bool|int)")
static

regex for extracting the type of the lambda function in metavariable function definition # noqa

Definition at line 75 of file test_variables_meta.py.

◆ registering_regex

re registering_regex
static
Initial value:
= re.compile(
r"(?s)REGISTER_METAVARIABLE.*?Manager::VariableDataType::(?:c_double|c_int|c_bool)" # noqa
)

regex for finding the REGISTER_METAVARIABLE statements.

Definition at line 67 of file test_variables_meta.py.


The documentation for this class was generated from the following file: