Class ListTokenSource

  • All Implemented Interfaces:
    TokenSource

    public class ListTokenSource
    extends java.lang.Object
    implements TokenSource
    Provides an implementation of TokenSource as a wrapper around a list of Token objects.

    If the final token in the list is an Token.EOF token, it will be used as the EOF token for every call to nextToken() after the end of the list is reached. Otherwise, an EOF token will be created.

    • Field Detail

      • tokens

        protected final java.util.List<? extends Token> tokens
        The wrapped collection of Token objects to return.
      • sourceName

        private final java.lang.String sourceName
        The name of the input source. If this value is null, a call to getSourceName() should return the source name used to create the the next token in tokens (or the previous token if the end of the input has been reached).
      • i

        protected int i
        The index into tokens of token to return by the next call to nextToken(). The end of the input is indicated by this value being greater than or equal to the number of items in tokens.
      • eofToken

        protected Token eofToken
        This field caches the EOF token for the token source.
    • Constructor Detail

      • ListTokenSource

        public ListTokenSource​(java.util.List<? extends Token> tokens)
        Constructs a new ListTokenSource instance from the specified collection of Token objects.
        Parameters:
        tokens - The collection of Token objects to provide as a TokenSource.
        Throws:
        java.lang.NullPointerException - if tokens is null
      • ListTokenSource

        public ListTokenSource​(java.util.List<? extends Token> tokens,
                               java.lang.String sourceName)
        Constructs a new ListTokenSource instance from the specified collection of Token objects and source name.
        Parameters:
        tokens - The collection of Token objects to provide as a TokenSource.
        sourceName - The name of the TokenSource. If this value is null, getSourceName() will attempt to infer the name from the next Token (or the previous token if the end of the input has been reached).
        Throws:
        java.lang.NullPointerException - if tokens is null
    • Method Detail

      • getCharPositionInLine

        public int getCharPositionInLine()
        Get the index into the current line for the current position in the input stream. The first character on a line has position 0.
        Specified by:
        getCharPositionInLine in interface TokenSource
        Returns:
        The line number for the current position in the input stream, or -1 if the current token source does not track character positions.
      • nextToken

        public Token nextToken()
        Return a Token object from your input stream (usually a CharStream). Do not fail/return upon lexing error; keep chewing on the characters until you get a good one; errors are not passed through to the parser.
        Specified by:
        nextToken in interface TokenSource
      • getLine

        public int getLine()
        Get the line number for the current position in the input stream. The first line in the input is line 1.
        Specified by:
        getLine in interface TokenSource
        Returns:
        The line number for the current position in the input stream, or 0 if the current token source does not track line numbers.
      • getInputStream

        public CharStream getInputStream()
        Get the CharStream from which this token source is currently providing tokens.
        Specified by:
        getInputStream in interface TokenSource
        Returns:
        The CharStream associated with the current position in the input, or null if no input stream is available for the token source.
      • getSourceName

        public java.lang.String getSourceName()
        Gets the name of the underlying input source. This method returns a non-null, non-empty string. If such a name is not known, this method returns IntStream.UNKNOWN_SOURCE_NAME.
        Specified by:
        getSourceName in interface TokenSource