class RMMSeg::SimpleAlgorithm
Public Class Methods
new(text, token=Token)
click to toggle source
Create a new SimpleAlgorithm
. The only rule used by this algorithm is MMRule
.
Calls superclass method
RMMSeg::Algorithm::new
# File lib/rmmseg/simple_algorithm.rb, line 10 def initialize(text, token=Token) super end
Public Instance Methods
get_cjk_word()
click to toggle source
Get the most proper CJK word.
# File lib/rmmseg/simple_algorithm.rb, line 15 def get_cjk_word dic = Dictionary.instance i = Config.max_word_length if i + @index > @chars.length i = @chars.length - @index end chars = @chars[@index, i] word = chars.join while i > 1 && !dic.has_word?(word) i -= 1 word.slice!(-chars[i].size,chars[i].size) # truncate last char end token = @token.new(word, @byte_index, @byte_index+word.size) @index += i @byte_index += word.size return token end