class Rley::Lexical::TokenRange

A token range (also called an extent) represents an interval of token positions that is matched by a given grammar symbol. For instance consider the expression E: 3 + 11, let’s assume that the integer literal ‘3’ is the fifth input token and that the ‘+’ and ‘11’ tokens are respectively at position 6 and 7; then the token range associated with E is [5, 7] While the parse tree/forest is being constructed the boundaries of the token range can be temporarily undefined (= set to nil)