Package org.antlr.v4.runtime
Class CommonToken
java.lang.Object
org.antlr.v4.runtime.CommonToken
- All Implemented Interfaces:
Serializable
,Token
,WritableToken
- Direct Known Subclasses:
TokenTagToken
- See Also:
-
Field Summary
FieldsModifier and TypeFieldDescriptionprotected int
This is the backing field forgetChannel()
andsetChannel(int)
.protected int
This is the backing field forgetCharPositionInLine()
andsetCharPositionInLine(int)
.protected static final Pair
<TokenSource, CharStream> protected int
This is the backing field forgetTokenIndex()
andsetTokenIndex(int)
.protected int
This is the backing field forgetLine()
andsetLine(int)
.protected Pair
<TokenSource, CharStream> This is the backing field forgetTokenSource()
andgetInputStream()
.protected int
This is the backing field forgetStartIndex()
andsetStartIndex(int)
.protected int
This is the backing field forgetStopIndex()
andsetStopIndex(int)
.protected String
This is the backing field forgetText()
when the token text is explicitly set in the constructor or viasetText(java.lang.String)
.protected int
This is the backing field forgetType()
andsetType(int)
.Fields inherited from interface org.antlr.v4.runtime.Token
DEFAULT_CHANNEL, EOF, EPSILON, HIDDEN_CHANNEL, INVALID_TYPE, MIN_USER_CHANNEL_VALUE, MIN_USER_TOKEN_TYPE
-
Constructor Summary
ConstructorsConstructorDescriptionCommonToken
(int type) Constructs a newCommonToken
with the specified token type.CommonToken
(int type, String text) Constructs a newCommonToken
with the specified token type and text.CommonToken
(Pair<TokenSource, CharStream> source, int type, int channel, int start, int stop) CommonToken
(Token oldToken) Constructs a newCommonToken
as a copy of anotherToken
. -
Method Summary
Modifier and TypeMethodDescriptionint
Return the channel this token.int
The index of the first character of this token relative to the beginning of the line at which it occurs, 0..n-1Gets theCharStream
from which this token was derived.int
getLine()
The line number on which the 1st character of this token was matched, line=1..nint
The starting character index of the token This method is optional; return -1 if not implemented.int
The last character index of the token.getText()
Get the text of the token.int
An index from 0..n-1 of the token object in the input stream.Gets theTokenSource
which created this token.int
getType()
Get the token type of the tokenvoid
setChannel
(int channel) void
setCharPositionInLine
(int charPositionInLine) void
setLine
(int line) void
setStartIndex
(int start) void
setStopIndex
(int stop) void
Explicitly set the text for this token.void
setTokenIndex
(int index) void
setType
(int type) toString()
toString
(Recognizer<?, ?> r)
-
Field Details
-
EMPTY_SOURCE
-
type
protected int typeThis is the backing field forgetType()
andsetType(int)
. -
line
protected int lineThis is the backing field forgetLine()
andsetLine(int)
. -
charPositionInLine
protected int charPositionInLineThis is the backing field forgetCharPositionInLine()
andsetCharPositionInLine(int)
. -
channel
protected int channelThis is the backing field forgetChannel()
andsetChannel(int)
. -
source
This is the backing field forgetTokenSource()
andgetInputStream()
.These properties share a field to reduce the memory footprint of
CommonToken
. Tokens created by aCommonTokenFactory
from the same source and input stream share a reference to the samePair
containing these values. -
text
This is the backing field forgetText()
when the token text is explicitly set in the constructor or viasetText(java.lang.String)
.- See Also:
-
index
protected int indexThis is the backing field forgetTokenIndex()
andsetTokenIndex(int)
. -
start
protected int startThis is the backing field forgetStartIndex()
andsetStartIndex(int)
. -
stop
protected int stopThis is the backing field forgetStopIndex()
andsetStopIndex(int)
.
-
-
Constructor Details
-
CommonToken
public CommonToken(int type) Constructs a newCommonToken
with the specified token type.- Parameters:
type
- The token type.
-
CommonToken
-
CommonToken
Constructs a newCommonToken
with the specified token type and text.- Parameters:
type
- The token type.text
- The text of the token.
-
CommonToken
Constructs a newCommonToken
as a copy of anotherToken
.If
oldToken
is also aCommonToken
instance, the newly constructed token will share a reference to thetext
field and thePair
stored insource
. Otherwise,text
will be assigned the result of callinggetText()
, andsource
will be constructed from the result ofToken.getTokenSource()
andToken.getInputStream()
.- Parameters:
oldToken
- The token to copy.
-
-
Method Details
-
getType
public int getType()Description copied from interface:Token
Get the token type of the token -
setLine
public void setLine(int line) - Specified by:
setLine
in interfaceWritableToken
-
getText
Description copied from interface:Token
Get the text of the token. -
setText
Explicitly set the text for this token. If {code text} is notnull
, thengetText()
will return this value rather than extracting the text from the input.- Specified by:
setText
in interfaceWritableToken
- Parameters:
text
- The explicit text of the token, ornull
if the text should be obtained from the input along with the start and stop indexes of the token.
-
getLine
public int getLine()Description copied from interface:Token
The line number on which the 1st character of this token was matched, line=1..n -
getCharPositionInLine
public int getCharPositionInLine()Description copied from interface:Token
The index of the first character of this token relative to the beginning of the line at which it occurs, 0..n-1- Specified by:
getCharPositionInLine
in interfaceToken
-
setCharPositionInLine
public void setCharPositionInLine(int charPositionInLine) - Specified by:
setCharPositionInLine
in interfaceWritableToken
-
getChannel
public int getChannel()Description copied from interface:Token
Return the channel this token. Each token can arrive at the parser on a different channel, but the parser only "tunes" to a single channel. The parser ignores everything not on DEFAULT_CHANNEL.- Specified by:
getChannel
in interfaceToken
-
setChannel
public void setChannel(int channel) - Specified by:
setChannel
in interfaceWritableToken
-
setType
public void setType(int type) - Specified by:
setType
in interfaceWritableToken
-
getStartIndex
public int getStartIndex()Description copied from interface:Token
The starting character index of the token This method is optional; return -1 if not implemented.- Specified by:
getStartIndex
in interfaceToken
-
setStartIndex
public void setStartIndex(int start) -
getStopIndex
public int getStopIndex()Description copied from interface:Token
The last character index of the token. This method is optional; return -1 if not implemented.- Specified by:
getStopIndex
in interfaceToken
-
setStopIndex
public void setStopIndex(int stop) -
getTokenIndex
public int getTokenIndex()Description copied from interface:Token
An index from 0..n-1 of the token object in the input stream. This must be valid in order to print token streams and use TokenRewriteStream. Return -1 to indicate that this token was conjured up since it doesn't have a valid index.- Specified by:
getTokenIndex
in interfaceToken
-
setTokenIndex
public void setTokenIndex(int index) - Specified by:
setTokenIndex
in interfaceWritableToken
-
getTokenSource
Description copied from interface:Token
Gets theTokenSource
which created this token.- Specified by:
getTokenSource
in interfaceToken
-
getInputStream
Description copied from interface:Token
Gets theCharStream
from which this token was derived.- Specified by:
getInputStream
in interfaceToken
-
toString
-
toString
-