class Rley::RGN::Tokenizer

A tokenizer for the Rley notation language. Responsibility: break input into a sequence of token objects. The tokenizer should recognize: Identifiers, Number literals including single digit String literals (quote delimited) Delimiters: e.g. parentheses ‘(’, ‘)’ Separators: e.g. comma