module Grammaphone::Token

Token contains methods that classify what kind of element type a specific rule pattern is.

Constants

LITERAL_PREFIX

The prefix used to denote a literal element.

Public Class Methods

backref?(element) click to toggle source

Checks if an element expects a backreference. A backreference is prefixed by an equals sign (“=”)

# File lib/grammaphone/tokens.rb, line 176
def self.backref?(element)
  element[0] == "=" 
end
clean_backref(element, matches) click to toggle source

Removes the denotative marks of a backreference, and returns a literal token that references a previous element referenced. The token is the exact token matched, regardless of if the original element was a literal or pattern.

# File lib/grammaphone/tokens.rb, line 184
def self.clean_backref(element, matches)
  matches[element[1..].to_i - 1]
end
clean_literal(token) click to toggle source

Removes the denotative marks of a literal, and returns the resulting value.

# File lib/grammaphone/tokens.rb, line 138
def self.clean_literal(token)
  token[1..]
end
clean_pattern(token) click to toggle source

Removes the denotative marks of a pattern, and returns a Regexp that matches the pattern exactly. That is, the pattern describes the whole token, and nothing less.

# File lib/grammaphone/tokens.rb, line 160
def self.clean_pattern(token)
  /\A#{token[1...-1]}\Z/
end
literal?(token) click to toggle source

Checks if an element expects a literal value. A literal element is denoted by being prefixed by the value of `LITERAL_PREFIX`.

# File lib/grammaphone/tokens.rb, line 133
def self.literal?(token)
  token[0] == LITERAL_PREFIX
end
matches_backref?(element, token, matches) click to toggle source

Returns whether the token is described by the element and that the element is a backreference. A token is described by a backreference if it matches the element at the 1-indexed position exactly.

A backreference cannot describe a rule, even if that rule describes 1 or fewer tokens.

# File lib/grammaphone/tokens.rb, line 194
def self.matches_backref?(element, token, matches)
  !token.nil? && backref?(element) && token == clean_backref(element, matches) 
end
matches_literal?(element, token) click to toggle source

Returns whether the token is described by the element and that the element is a literal.

Returns `false` if the token is `nil`, since it's impossible to match a literal `nil`. Note, `nil` differs from an empty token.

# File lib/grammaphone/tokens.rb, line 147
def self.matches_literal?(element, token)
  !token.nil? && literal?(element) && token == clean_literal(element)
end
matches_pattern?(element, token) click to toggle source

Returns whether the token is described by the element and that the element is a pattern.

Returns `false` if the token is `nil`, and the pattern doesn't match the empty string.

# File lib/grammaphone/tokens.rb, line 169
def self.matches_pattern?(element, token)
  pattern?(element) && (token =~ clean_pattern(element)) ||
    token.nil? && "" =~ clean_pattern(element)
end
pattern?(token) click to toggle source

Checks if an element expects a pattern value. A pattern element is denoted by being surrounded by forward slashes.

# File lib/grammaphone/tokens.rb, line 153
def self.pattern?(token)
  token[0] == "/" && token[-1] == "/"
end