class RMMSeg::SimpleAlgorithm

Public Class Methods

new(text, token=Token) click to toggle source

Create a new SimpleAlgorithm . The only rule used by this algorithm is MMRule .

Calls superclass method RMMSeg::Algorithm::new
# File lib/rmmseg/simple_algorithm.rb, line 10
def initialize(text, token=Token)
  super
end

Public Instance Methods

get_cjk_word() click to toggle source

Get the most proper CJK word.

# File lib/rmmseg/simple_algorithm.rb, line 15
def get_cjk_word
  dic = Dictionary.instance
  i = Config.max_word_length
  if i + @index > @chars.length
    i = @chars.length - @index
  end
  chars = @chars[@index, i]
  word = chars.join

  while i > 1 && !dic.has_word?(word)
    i -= 1
    word.slice!(-chars[i].size,chars[i].size) # truncate last char
  end

  token = @token.new(word, @byte_index, @byte_index+word.size)

  @index += i
  @byte_index += word.size

  return token
end