Testing Tools
Contents
4.3. Testing Tools#
These tools are used to run the continuous integration tests. We have unittests and script tests.
4.3.1. b2test-units
: Run unittests#
This program runs all unittests of the Belle2 Software. It is based on Google
Test. Usually it’s enough to call it
without arguments. Here we just document the most useful arguments, please use
--help
to get a more detailed list of available options.
Optional Arguments
- --help
Show a detailed list of all options
- --gtest_list_tests
List all tests instead of running them
- --gtest_filter=PATTERN
Run only the tests whose name matches the pattern
- --gtest_output_xml=FILENAME
Write the test results into a JUnit compatible XML file
4.3.2. b2test-scripts
: Run framework tests#
Run basf2 framework tests of the given packages. If no argument is given all tests are run.
For each package execute all files ending in *.py
inside the tests/
directory as test cases. A return value of 0 is treaded as success, other return
values are treated as failures. In addition if *.out
or *.err
files with
the same base name are present in the tests directory they will be treated as
expected output on stdout or stderr respectively. If the output of the script
differes from the contents of these files the test is marked as failed as well.
The first line from stdout will not be compared to allow executing the scripts
from a different directory.
Note
Tests can decide that they need to be skipped under certain conditions. To do so
a test script can write a line beginning with TEST SKIPPED:
followed by the
reason for skipping on stderr and exit with a non-zero return code.
See also
usage: b2test-scripts [-h] [-j N] [--xml XML] [-n DIFF_LINES] [-q]
[--filter FILTER] [--exclude EXCLUDE] [-t SECONDS]
[DIRECTORY [DIRECTORY ...]]
Required Arguments
- DIRECTORY
run all tests included in this directory.If none are specified run all tests
Optional Arguments
- -j
Number of test cases to run in parallel
Default: 1
- --xml
Save results into given file using xml format compatible with JUnit parsers
- -n, --diff-lines
Number of lines to show if output is different from template
Default: 10
- -q, --quiet
if given don’t print output of failed tests
- --filter
Only run tests where the name of the steering file matches the given regular expression
- --exclude
Exclude all tests where the name of the steering file matches the given regular expression
- -t, --timeout
if given this is the maximum runtime in seconds for each test case before they are killed and marked as failed
Examples
run all tests with 8 tests being run in parallel:
$ b2test-scripts -j8
run the framework tests and save the output as XML:
$ b2test-scripts --xml=framework-tests.xml framework
run only tests which where the python filename contains seqroot:
$ b2test-scripts --filter=seqroot