class RMMSeg::Token
A Token
consists of a term's text and the start and end offset of the term.
Attributes
end[RW]
The one greater than the position of the last byte of the token. This is byte index instead of character.
start[RW]
The start position of the token. This is byte index instead of character.
text[RW]
The text of the token
Public Class Methods
new(text, start_pos, end_pos)
click to toggle source
text
is the ref to the whole text. In other words: text[start_pos...end_pos]
should be the string held by this token.
# File lib/rmmseg/token.rb, line 19 def initialize(text, start_pos, end_pos) @text = text @start = start_pos @end = end_pos end
Public Instance Methods
to_s()
click to toggle source
# File lib/rmmseg/token.rb, line 25 def to_s @text.dup end