Belle II Software development
Public Member Functions | Public Attributes | Static Public Attributes | List of all members
B2Lexer Class Reference
Inheritance diagram for B2Lexer:

Public Member Functions

def __init__ (self)
 
def LBRACK (self, t)
 
def RBRACK (self, t)
 
def LPAREN (self, t)
 

Public Attributes

 control_token_stack
 control_token_stack (list): stack for keeping track of seen brackets and parenthesis.
 

Static Public Attributes

dict cut_tokens
 cut specific tokens
 
dict expression_tokens
 expression tokens, also needed for cut.
 
dict tokens = expression_tokens.union(cut_tokens)
 Set of all tokens.
 
str ignore = " \t\n"
 ignore spaces tabs and newlines
 
dict literals = {r","}
 comma token definition as literal
 
str EQUALEQUAL = r"=="
 token regular expression for '=='
 
str GREATEREQUAL = r">="
 token regular expression for '>='
 
str LESSEQUAL = r"<="
 token regular expression for '<='
 
str GREATER = r">"
 token regular expression for '>'
 
str LESS = r"<"
 token regular expression for '<'
 
str NOTEQUAL = r"!="
 token regular expression for '!='
 
str POWER = r"\*\*|\^"
 token regular expression for power, both '**' and '^' allowed
 
str TIMES = r"\*"
 token regular expression for '*'
 
str DIVIDE = r"/"
 token regular expression for '/'
 
str PLUS = r"\+"
 token regular expression for '+'
 
str MINUS = r"-"
 token regular expression for '-'
 
 Parameters :
 
 Raises :
 
int SyntaxError : 1. If control_token_stack is empty, which means
 
 Returns :
 
 Warning :
 

Detailed Description

Class responsible for scanning the cut and generating a stream of tokens.
The token stream can be passed to `B2Parser` to generate a syntax tree.

Definition at line 73 of file b2parser.py.

Constructor & Destructor Documentation

◆ __init__()

def __init__ (   self) 
Initialize Lexer

Definition at line 79 of file b2parser.py.

79 def __init__(self):
80 """Initialize Lexer"""
81
84 self.control_token_stack = list()
85

Member Function Documentation

◆ LBRACK()

def LBRACK (   self,
  t 
) 
Scan opening bracket.

Parameters:
    t (sly.lex.token): token of type LBRACK

Raises:
    SyntaxError: if no following closing bracket is found
        in the input.

Side Effect:
    Pushes 'BRACK' onto control_token_stack

Returns:
    sly.lex.Token

Definition at line 146 of file b2parser.py.

146 def LBRACK(self, t):
147 """
148 Scan opening bracket.
149
150 Parameters:
151 t (sly.lex.token): token of type LBRACK
152
153 Raises:
154 SyntaxError: if no following closing bracket is found
155 in the input.
156
157 Side Effect:
158 Pushes 'BRACK' onto control_token_stack
159
160 Returns:
161 sly.lex.Token
162 """
163 if "]" not in self.text[self.index:]:
164 raise SyntaxError("Unmatched '[' in cut.")
165 self.control_token_stack.append("BRACK")
166 return t
167

◆ LPAREN()

def LPAREN (   self,
  t 
) 
Scan opening parenthesis.

Parameters:
    t (sly.lex.token): token of type LPAREN

Raises:
    SyntaxError: if no following closing parenthesis is found
        in the input.

Side Effect:
    Pushes 'PAREN' onto control_token_stack

Returns:
    sly.lex.Token

Definition at line 198 of file b2parser.py.

198 def LPAREN(self, t):
199 """
200 Scan opening parenthesis.
201
202 Parameters:
203 t (sly.lex.token): token of type LPAREN
204
205 Raises:
206 SyntaxError: if no following closing parenthesis is found
207 in the input.
208
209 Side Effect:
210 Pushes 'PAREN' onto control_token_stack
211
212 Returns:
213 sly.lex.Token
214 """
215 if ")" not in self.text[self.index:]:
216 raise SyntaxError("Unmatched '('")
217 self.control_token_stack.append("PAREN")
218 return t
219

◆ RBRACK()

def RBRACK (   self,
  t 
) 
Scan closing bracket.

Parameters:
    t (sly.lex.token): token of type RBRACK

Raises:
    SyntaxError: 1. If control_token_stack is empty, which means
        no bracket was opened previously.
                 2. If state of control_token_stack is 'PAREN', which
        means a closing parenthesis is expected.

Side Effect:
    Pops object from control_token_stack

Returns:
    sly.lex.Token

Definition at line 169 of file b2parser.py.

169 def RBRACK(self, t):
170 """
171 Scan closing bracket.
172
173 Parameters:
174 t (sly.lex.token): token of type RBRACK
175
176 Raises:
177 SyntaxError: 1. If control_token_stack is empty, which means
178 no bracket was opened previously.
179 2. If state of control_token_stack is 'PAREN', which
180 means a closing parenthesis is expected.
181
182 Side Effect:
183 Pops object from control_token_stack
184
185 Returns:
186 sly.lex.Token
187 """
188 try:
189 state = self.control_token_stack.pop()
190 except IndexError: # pop from empty list
191 raise SyntaxError("Unmatched ']' in cut.")
192 if state == "BRACK":
193 return t
194 elif state == "PAREN":
195 raise SyntaxError("Illegal ']', expected ')'.")
196

Member Data Documentation

◆ control_token_stack

control_token_stack

control_token_stack (list): stack for keeping track of seen brackets and parenthesis.

Allows finding parenthesis and bracket syntax errors on scanner level.

Definition at line 84 of file b2parser.py.

◆ cut_tokens

dict cut_tokens
static
Initial value:
= {
# structure tokens
RBRACK, LBRACK, # noqa: F821
# boolean operators
AND, OR, NOT, # noqa: F821
# comparison operators
EQUALEQUAL, GREATEREQUAL, LESSEQUAL, GREATER, LESS, # noqa: F821
NOTEQUAL, # noqa: F821
}

cut specific tokens

Definition at line 87 of file b2parser.py.

◆ DIVIDE

str DIVIDE = r"/"
static

token regular expression for '/'

Definition at line 135 of file b2parser.py.

◆ EQUALEQUAL

str EQUALEQUAL = r"=="
static

token regular expression for '=='

Definition at line 117 of file b2parser.py.

◆ expression_tokens

dict expression_tokens
static
Initial value:
= {
LPAREN, RPAREN, # noqa: F821
# data types
DOUBLE, INTEGER, IDENTIFIER, BOOLEAN, # noqa: F821
# arithmetic operators
POWER, TIMES, DIVIDE, PLUS, MINUS # noqa: F821
}

expression tokens, also needed for cut.

Definition at line 97 of file b2parser.py.

◆ GREATER

str GREATER = r">"
static

token regular expression for '>'

Definition at line 123 of file b2parser.py.

◆ GREATEREQUAL

str GREATEREQUAL = r">="
static

token regular expression for '>='

Definition at line 119 of file b2parser.py.

◆ ignore

str ignore = " \t\n"
static

ignore spaces tabs and newlines

Definition at line 109 of file b2parser.py.

◆ LESS

str LESS = r"<"
static

token regular expression for '<'

Definition at line 125 of file b2parser.py.

◆ LESSEQUAL

str LESSEQUAL = r"<="
static

token regular expression for '<='

Definition at line 121 of file b2parser.py.

◆ literals

dict literals = {r","}
static

comma token definition as literal

Definition at line 113 of file b2parser.py.

◆ MINUS

str MINUS = r"-"
static

token regular expression for '-'

Definition at line 139 of file b2parser.py.

◆ NOTEQUAL

str NOTEQUAL = r"!="
static

token regular expression for '!='

Definition at line 127 of file b2parser.py.

◆ Parameters

Parameters :
static

Definition at line 225 of file b2parser.py.

◆ PLUS

str PLUS = r"\+"
static

token regular expression for '+'

Definition at line 137 of file b2parser.py.

◆ POWER

str POWER = r"\*\*|\^"
static

token regular expression for power, both '**' and '^' allowed

Definition at line 131 of file b2parser.py.

◆ Raises

Raises :
static

Definition at line 228 of file b2parser.py.

◆ Returns

Returns :
static

Definition at line 237 of file b2parser.py.

◆ SyntaxError

int SyntaxError : 1. If control_token_stack is empty, which means
static

Definition at line 229 of file b2parser.py.

◆ TIMES

str TIMES = r"\*"
static

token regular expression for '*'

Definition at line 133 of file b2parser.py.

◆ tokens

dict tokens = expression_tokens.union(cut_tokens)
static

Set of all tokens.

Definition at line 105 of file b2parser.py.

◆ Warning

Warning :
static

Definition at line 284 of file b2parser.py.

The documentation for this class was generated from the following file: