Package com.ibm.icu.impl.data
Class TokenIterator
- java.lang.Object
-
- com.ibm.icu.impl.data.TokenIterator
-
public class TokenIterator extends java.lang.Object
An iterator class that returns successive string tokens from some source. String tokens are, in general, separated by Pattern_White_Space in the source test. Furthermore, they may be delimited by either single or double quotes (opening and closing quotes must match). Escapes are processed using standard ICU unescaping.2015-sep-03 TODO: Only used in com.ibm.icu.dev.test.format, move there.
-
-
Constructor Summary
Constructors Constructor Description TokenIterator(ResourceReader r)
Construct an iterator over the tokens returned by the given ResourceReader, ignoring blank lines and comment lines (first non-blank character is '#').
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description java.lang.String
describePosition()
Return a string description of the position of the last line returned by readLine() or readLineSkippingComments().int
getLineNumber()
Return the one-based line number of the line of the last token returned by next().java.lang.String
next()
Return the next token from this iterator, or null if the last token has been returned.private int
nextToken(int position)
Read the next token from 'this.line' and append it to 'this.buf'.
-
-
-
Field Detail
-
reader
private ResourceReader reader
-
line
private java.lang.String line
-
buf
private java.lang.StringBuffer buf
-
done
private boolean done
-
pos
private int pos
-
lastpos
private int lastpos
-
-
Constructor Detail
-
TokenIterator
public TokenIterator(ResourceReader r)
Construct an iterator over the tokens returned by the given ResourceReader, ignoring blank lines and comment lines (first non-blank character is '#'). Note that trailing comments on a line, beginning with the first unquoted '#', are recognized.
-
-
Method Detail
-
next
public java.lang.String next() throws java.io.IOException
Return the next token from this iterator, or null if the last token has been returned.- Throws:
java.io.IOException
-
getLineNumber
public int getLineNumber()
Return the one-based line number of the line of the last token returned by next(). Should only be called after a call to next(); otherwise the return value is undefined.
-
describePosition
public java.lang.String describePosition()
Return a string description of the position of the last line returned by readLine() or readLineSkippingComments().
-
nextToken
private int nextToken(int position)
Read the next token from 'this.line' and append it to 'this.buf'. Tokens are separated by Pattern_White_Space. Tokens may also be delimited by double or single quotes. The closing quote must match the opening quote. If a '#' is encountered, the rest of the line is ignored, unless it is backslash-escaped or within quotes.- Parameters:
position
- the offset into the string- Returns:
- offset to the next character to read from line, or if the end of the line is reached without scanning a valid token, -1
-
-