class RMMSeg::Ferret::PunctuationFilter

PunctuationFilter filter out the stand alone Chinese punctuation tokens.

Public Class Methods

new(stream) click to toggle source
# File lib/rmmseg/ferret.rb, line 84
def initialize(stream)
  @stream = stream
end

Public Instance Methods

next() click to toggle source

Get next token, skip stand alone Chinese punctuations.

# File lib/rmmseg/ferret.rb, line 89
def next
  token = @stream.next
  dic = Dictionary.instance

  until token.nil? || !(dic.include? token.text)
    token = @stream.next
  end

  token
end
text() click to toggle source
# File lib/rmmseg/ferret.rb, line 100
def text
  @stream.text
end
text=(str) click to toggle source
# File lib/rmmseg/ferret.rb, line 104
def text=(str)
  @stream.text = str
end