finesse.script.containers.KatMetaToken

Overview

class finesse.script.containers.KatMetaToken(lineno: int, start_index: int, stop_index: int, type: str)

Bases: TokenContainer

A token that may not map to a real token.

Meta tokens are employed by KatTokenizer when converting whitespace and newline tokens (provided by KatRawTokenizer) used as delimiters into DELIMITER. The use of a single meta token for DELIMITER in this case simplifies the parser rules.

By definition, the value of a meta token does not matter so the class doesn’t support having one. Similarly, to help prevent bugs, an exception is raised if the token is attempted to be displayed.