finesse.script.highlighter.KatScriptPygmentsLexer.get_tokens_unprocessed¶
- KatScriptPygmentsLexer.get_tokens_unprocessed(text)[source]¶
This method should process the text and return an iterable of
(index, tokentype, value)
tuples whereindex
is the starting position of the token within the input text.It must be overridden by subclasses. It is recommended to implement it as a generator to maximize effectiveness.