class RMMSeg::Token

A Token consists of a term’s text and the start and end offset of the term.

Attributes

end[RW]

The one greater than the position of the last byte of the token. This is byte index instead of character.

start[RW]

The start position of the token. This is byte index instead of character.

text[RW]

The text of the token

Public Class Methods

new(text, start_pos, end_pos) click to toggle source

text is the ref to the whole text. In other words: text[start_pos...end_pos] should be the string held by this token.

# File lib/rmmseg/token.rb, line 19
def initialize(text, start_pos, end_pos)
  @text  = text
  @start = start_pos
  @end   = end_pos
end

Public Instance Methods

to_s() click to toggle source
# File lib/rmmseg/token.rb, line 25
def to_s
  @text.dup
end