Inheritance diagram for B2Lexer:

Public Member Functions
	__init__ (self)

	LBRACK (self, t)

	RBRACK (self, t)

	LPAREN (self, t)

	RPAREN (self, t)

	DOUBLE (self, t)

	INTEGER (self, t)

	IDENTIFIER (self, t)

Public Attributes
	control_token_stack = list()
	control_token_stack (list): stack for keeping track of seen brackets and parenthesis.

]	index

Static Public Attributes
dict	cut_tokens
	cut specific tokens

dict	expression_tokens
	expression tokens, also needed for cut.

dict	tokens = expression_tokens.union(cut_tokens)
	Set of all tokens.

str	ignore = " \t\n"
	ignore spaces tabs and newlines

dict	literals = {r","}
	comma token definition as literal

str	EQUALEQUAL = r"=="
	token regular expression for '=='

str	GREATEREQUAL = r">="
	token regular expression for '>='

str	LESSEQUAL = r"<="
	token regular expression for '<='

str	GREATER = r">"
	token regular expression for '>'

str	LESS = r"<"
	token regular expression for '<'

str	NOTEQUAL = r"!="
	token regular expression for '!='

str	POWER = r"\\\|\^"
	token regular expression for power, both '**' and '^' allowed

str	TIMES = r"\*"
	token regular expression for '*'

str	DIVIDE = r"/"
	token regular expression for '/'

str	PLUS = r"\+"
	token regular expression for '+'

str	MINUS = r"-"
	token regular expression for '-'

Detailed Description

Class responsible for scanning the cut and generating a stream of tokens.
The token stream can be passed to `B2Parser` to generate a syntax tree.

Definition at line 73 of file b2parser.py.

Constructor & Destructor Documentation

◆ init()

__init__ ( self )


Initialize Lexer

Definition at line 79 of file b2parser.py.

    def __init__(self):
        """Initialize Lexer"""
        ## control_token_stack (list): stack for keeping track of seen brackets
        ## and parenthesis. Allows finding parenthesis and bracket syntax
        ## errors on scanner level.
        self.control_token_stack = list()
 

Member Function Documentation

◆ DOUBLE()

DOUBLE	(		self,
			t )

Scanning function for double values

Parameters:
    t (sly.lex.Token): initial token generated by the scanner library.
        The value attribute is of type str initially, equals
        the matched sequence and is casted to float.

Possible notations covered by this regular expression:
    Normal decimal notation e.g 0.1
    Hanging decimal separator notation e.g 1.
    Preceding decimal separator notation e.g .1
    Scientific notation with (signed) exponents e.g 1.0E4, 1.e-4, .1E+3
    Exponents are case insensitive e.g 1.e4, 1.E4
    Integer with exponent e.g 1E4

Returns:
    sly.lex.Token

Definition at line 250 of file b2parser.py.

    def DOUBLE(self, t):
        """
        Scanning function for double values
 
        Parameters:
            t (sly.lex.Token): initial token generated by the scanner library.
                The value attribute is of type str initially, equals
                the matched sequence and is casted to float.
 
        Possible notations covered by this regular expression:
            Normal decimal notation e.g 0.1
            Hanging decimal separator notation e.g 1.
            Preceding decimal separator notation e.g .1
            Scientific notation with (signed) exponents e.g 1.0E4, 1.e-4, .1E+3
            Exponents are case insensitive e.g 1.e4, 1.E4
            Integer with exponent e.g 1E4
 
        Returns:
            sly.lex.Token
        """
        t.value = float(t.value)
        return t
 

◆ IDENTIFIER()

IDENTIFIER	(		self,
			t )

Scanning function for identifiers

If a matched sequence equals reserved keywords of other tokens
the token type and value is remapped via the reserved dictionary.

Parameters:
    t (sly.lex.Token): initial token generated by the scanner library.
        value attribute equals the matched sequence.

Returns:
    sly.lex.Token

Definition at line 302 of file b2parser.py.

    def IDENTIFIER(self, t):
        """
        Scanning function for identifiers
 
        If a matched sequence equals reserved keywords of other tokens
        the token type and value is remapped via the reserved dictionary.
 
        Parameters:
            t (sly.lex.Token): initial token generated by the scanner library.
                value attribute equals the matched sequence.
 
        Returns:
            sly.lex.Token
        """
        reserved = {
            "and": "AND",
            "or": "OR",
            "not": "NOT",
            "True": "BOOLEAN",
            "true": "BOOLEAN",
            "False": "BOOLEAN",
            "false": "BOOLEAN",
            "nan": "DOUBLE",
            "infinity": "DOUBLE",
            "inf": "DOUBLE",
        }
        # Check for reserved words
        t.type = reserved.get(t.value, "IDENTIFIER")
 
        # Set value to bool if BOOLEAN type was returned from reserved dict.
        if t.type == "BOOLEAN":
            t.value = t.value == "True" or t.value == "true"
        # Take care of special infinity and nan values.
        if t.type == "DOUBLE":
            t.value = float(t.value)
        if t.type == "IDENTIFIER":
            try:
                if self.text[self.index] == "(":
                    # Check that closing parenthesis exists
                    if ")" not in self.text[self.index:]:
                        raise SyntaxError("Unmatched '('")
                    else:
                        self.push_state(B2ParameterLexer)
            except IndexError:
                pass
        return t
 
 

◆ INTEGER()

INTEGER	(		self,
			t )

Scanning function for integer values
Allows normal and hex notation (case insensitive)

Parameters:
    t (sly.lex.Token): initial token generated by the scanner library.
        The value attribute is of type str initially, equals
        the matched sequence and is casted to int.

Warning:
    python int-objects are converted
    to the standard c++ int datatype (32bit).
    Overflows can happen because numerical limits
    of python int and c++ int datatypes differ.
    If you need to input large values write it as double.

Returns:
    sly.lex.Token

Definition at line 274 of file b2parser.py.

    def INTEGER(self, t):
        """
        Scanning function for integer values
        Allows normal and hex notation (case insensitive)
 
        Parameters:
            t (sly.lex.Token): initial token generated by the scanner library.
                The value attribute is of type str initially, equals
                the matched sequence and is casted to int.
 
        Warning:
            python int-objects are converted
            to the standard c++ int datatype (32bit).
            Overflows can happen because numerical limits
            of python int and c++ int datatypes differ.
            If you need to input large values write it as double.
 
        Returns:
            sly.lex.Token
        """
        try:
            t.value = int(t.value)
        except ValueError:
            # casting hex notation
            t.value = int(t.value, base=16)
        return t
 

◆ LBRACK()

LBRACK	(		self,
			t )

Scan opening bracket.

Parameters:
    t (sly.lex.token): token of type LBRACK

Raises:
    SyntaxError: if no following closing bracket is found
        in the input.

Side Effect:
    Pushes 'BRACK' onto control_token_stack

Returns:
    sly.lex.Token

Definition at line 146 of file b2parser.py.

    def LBRACK(self, t):
        """
        Scan opening bracket.
 
        Parameters:
            t (sly.lex.token): token of type LBRACK
 
        Raises:
            SyntaxError: if no following closing bracket is found
                in the input.
 
        Side Effect:
            Pushes 'BRACK' onto control_token_stack
 
        Returns:
            sly.lex.Token
        """
        if "]" not in self.text[self.index:]:
            raise SyntaxError("Unmatched '[' in cut.")
        self.control_token_stack.append("BRACK")
        return t
 

◆ LPAREN()

LPAREN	(		self,
			t )

Scan opening parenthesis.

Parameters:
    t (sly.lex.token): token of type LPAREN

Raises:
    SyntaxError: if no following closing parenthesis is found
        in the input.

Side Effect:
    Pushes 'PAREN' onto control_token_stack

Returns:
    sly.lex.Token

Definition at line 198 of file b2parser.py.

    def LPAREN(self, t):
        """
        Scan opening parenthesis.
 
        Parameters:
            t (sly.lex.token): token of type LPAREN
 
        Raises:
            SyntaxError: if no following closing parenthesis is found
                in the input.
 
        Side Effect:
            Pushes 'PAREN' onto control_token_stack
 
        Returns:
            sly.lex.Token
        """
        if ")" not in self.text[self.index:]:
            raise SyntaxError("Unmatched '('")
        self.control_token_stack.append("PAREN")
        return t
 

◆ RBRACK()

RBRACK	(		self,
			t )

Scan closing bracket.

Parameters:
    t (sly.lex.token): token of type RBRACK

Raises:
    SyntaxError: 1. If control_token_stack is empty, which means
        no bracket was opened previously.
                 2. If state of control_token_stack is 'PAREN', which
        means a closing parenthesis is expected.

Side Effect:
    Pops object from control_token_stack

Returns:
    sly.lex.Token

Definition at line 169 of file b2parser.py.

    def RBRACK(self, t):
        """
        Scan closing bracket.
 
        Parameters:
            t (sly.lex.token): token of type RBRACK
 
        Raises:
            SyntaxError: 1. If control_token_stack is empty, which means
                no bracket was opened previously.
                         2. If state of control_token_stack is 'PAREN', which
                means a closing parenthesis is expected.
 
        Side Effect:
            Pops object from control_token_stack
 
        Returns:
            sly.lex.Token
        """
        try:
            state = self.control_token_stack.pop()
        except IndexError:  # pop from empty list
            raise SyntaxError("Unmatched ']' in cut.")
        if state == "BRACK":
            return t
        elif state == "PAREN":
            raise SyntaxError("Illegal ']', expected ')'.")
 

◆ RPAREN()

RPAREN	(		self,
			t )

Scan closing parenthesis.

Parameters:
    t (sly.lex.token): token of type RPAREN

Raises:
    SyntaxError: 1. If control_token_stack is empty, which means
        no parenthesis was opened previously.
                 2. If state of control_token_stack is 'BRACK', which
        means a closing bracket is expected.

Side Effect:
    Pops state from control_token_stack

Returns:
    sly.lex.Token

Definition at line 221 of file b2parser.py.

    def RPAREN(self, t):
        """
        Scan closing parenthesis.
 
        Parameters:
            t (sly.lex.token): token of type RPAREN
 
        Raises:
            SyntaxError: 1. If control_token_stack is empty, which means
                no parenthesis was opened previously.
                         2. If state of control_token_stack is 'BRACK', which
                means a closing bracket is expected.
 
        Side Effect:
            Pops state from control_token_stack
 
        Returns:
            sly.lex.Token
        """
        try:
            state = self.control_token_stack.pop()
        except IndexError:  # pop from empty list
            raise SyntaxError("Unmatched ')' in cut.")
        if state == "BRACK":
            raise SyntaxError("Illegal ')', expected ']'.")
        elif state == "PAREN":
            return t
 

Member Data Documentation

◆ control_token_stack

control_token_stack = list()

control_token_stack (list): stack for keeping track of seen brackets and parenthesis.

Allows finding parenthesis and bracket syntax errors on scanner level.

Definition at line 84 of file b2parser.py.

◆ cut_tokens

dict cut_tokens

static

Initial value:

=  {
        # structure tokens
        RBRACK, LBRACK,  # noqa: F821
        # boolean operators
        AND, OR, NOT,  # noqa: F821
        # comparison operators
        EQUALEQUAL, GREATEREQUAL, LESSEQUAL, GREATER, LESS,  # noqa: F821
        NOTEQUAL,  # noqa: F821
    }

cut specific tokens

Definition at line 87 of file b2parser.py.

◆ DIVIDE

str DIVIDE = r"/"

static

token regular expression for '/'

Definition at line 135 of file b2parser.py.

◆ EQUALEQUAL

str EQUALEQUAL = r"=="

static

token regular expression for '=='

Definition at line 117 of file b2parser.py.

◆ expression_tokens

dict expression_tokens

static

Initial value:

=  {
        LPAREN, RPAREN,  # noqa: F821
        # data types
        DOUBLE, INTEGER, IDENTIFIER, BOOLEAN,  # noqa: F821
        # arithmetic operators
        POWER, TIMES, DIVIDE, PLUS, MINUS  # noqa: F821
    }

expression tokens, also needed for cut.

Definition at line 97 of file b2parser.py.

◆ GREATER

str GREATER = r">"

static

token regular expression for '>'

Definition at line 123 of file b2parser.py.

◆ GREATEREQUAL

str GREATEREQUAL = r">="

static

token regular expression for '>='

Definition at line 119 of file b2parser.py.

◆ ignore

str ignore = " \t\n"

static

ignore spaces tabs and newlines

Definition at line 109 of file b2parser.py.

◆ index

] index

Definition at line 163 of file b2parser.py.

◆ LESS

str LESS = r"<"

static

token regular expression for '<'

Definition at line 125 of file b2parser.py.

◆ LESSEQUAL

str LESSEQUAL = r"<="

static

token regular expression for '<='

Definition at line 121 of file b2parser.py.

◆ literals

dict literals = {r","}

static

comma token definition as literal

Definition at line 113 of file b2parser.py.

◆ MINUS

str MINUS = r"-"

static

token regular expression for '-'

Definition at line 139 of file b2parser.py.

◆ NOTEQUAL

str NOTEQUAL = r"!="

static

token regular expression for '!='

Definition at line 127 of file b2parser.py.

◆ PLUS

str PLUS = r"\+"

static

token regular expression for '+'

Definition at line 137 of file b2parser.py.

◆ POWER

str POWER = r"\*\*|\^"

static

token regular expression for power, both '**' and '^' allowed

Definition at line 131 of file b2parser.py.

◆ TIMES

str TIMES = r"\*"

static

token regular expression for '*'

Definition at line 133 of file b2parser.py.

◆ tokens

dict tokens = expression_tokens.union(cut_tokens)

static

Set of all tokens.

Definition at line 105 of file b2parser.py.

The documentation for this class was generated from the following file:

framework/scripts/b2parser.py

Public Member Functions

Public Attributes

Static Public Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ DOUBLE()

◆ IDENTIFIER()

◆ INTEGER()

◆ LBRACK()

◆ LPAREN()

◆ RBRACK()

◆ RPAREN()

Member Data Documentation

◆ control_token_stack

◆ cut_tokens

◆ DIVIDE

◆ EQUALEQUAL

◆ expression_tokens

◆ GREATER

◆ GREATEREQUAL

◆ ignore

◆ index

◆ LESS

◆ LESSEQUAL

◆ literals

◆ MINUS

◆ NOTEQUAL

◆ PLUS

◆ POWER

◆ TIMES

◆ tokens

◆ init()