class Rakko::Token

`Token` is a terminal symbol in grammer context.

Its `value` attribute represents token's value and its syntactic category. If `Token#value` is an instance of `Symbol`, its syntactic category is `Ident`.

Attributes

columnno[R]
lineno[R]
value[R]

Public Class Methods

new(value, columnno: 0, lineno: 0) click to toggle source

Creates a new Token instance.

@param value token's value. If the value is `String`, it will be duplicated and frozen. @param columnno [Integer] The column number of token position. number is `0` based. @param lineno [Integer] The line number of token position. number is `0` based.

# File lib/rakko/token.rb, line 21
def initialize(value, columnno: 0, lineno: 0)
  @columnno = columnno
  @lineno = lineno
  @value =
    case value
    when String
      value.dup.freeze
    else
      value
    end
end