module Grammaphone::Token
Token
contains methods that classify what kind of element type a specific rule pattern is.
Constants
- LITERAL_PREFIX
The prefix used to denote a literal element.
Public Class Methods
Checks if an element expects a backreference. A backreference is prefixed by an equals sign (“=”)
# File lib/grammaphone/tokens.rb, line 176 def self.backref?(element) element[0] == "=" end
Removes the denotative marks of a backreference, and returns a literal token that references a previous element referenced. The token is the exact token matched, regardless of if the original element was a literal or pattern.
# File lib/grammaphone/tokens.rb, line 184 def self.clean_backref(element, matches) matches[element[1..].to_i - 1] end
Removes the denotative marks of a literal, and returns the resulting value.
# File lib/grammaphone/tokens.rb, line 138 def self.clean_literal(token) token[1..] end
Removes the denotative marks of a pattern, and returns a Regexp that matches the pattern exactly. That is, the pattern describes the whole token, and nothing less.
# File lib/grammaphone/tokens.rb, line 160 def self.clean_pattern(token) /\A#{token[1...-1]}\Z/ end
Checks if an element expects a literal value. A literal element is denoted by being prefixed by the value of `LITERAL_PREFIX`.
# File lib/grammaphone/tokens.rb, line 133 def self.literal?(token) token[0] == LITERAL_PREFIX end
Returns whether the token is described by the element and that the element is a backreference. A token is described by a backreference if it matches the element at the 1-indexed position exactly.
A backreference cannot describe a rule, even if that rule describes 1 or fewer tokens.
# File lib/grammaphone/tokens.rb, line 194 def self.matches_backref?(element, token, matches) !token.nil? && backref?(element) && token == clean_backref(element, matches) end
Returns whether the token is described by the element and that the element is a literal.
Returns `false` if the token is `nil`, since it's impossible to match a literal `nil`. Note, `nil` differs from an empty token.
# File lib/grammaphone/tokens.rb, line 147 def self.matches_literal?(element, token) !token.nil? && literal?(element) && token == clean_literal(element) end
Returns whether the token is described by the element and that the element is a pattern.
Returns `false` if the token is `nil`, and the pattern doesn't match the empty string.
# File lib/grammaphone/tokens.rb, line 169 def self.matches_pattern?(element, token) pattern?(element) && (token =~ clean_pattern(element)) || token.nil? && "" =~ clean_pattern(element) end
Checks if an element expects a pattern value. A pattern element is denoted by being surrounded by forward slashes.
# File lib/grammaphone/tokens.rb, line 153 def self.pattern?(token) token[0] == "/" && token[-1] == "/" end