org.jruby.lexer.yacc
Class RubyYaccLexer

java.lang.Object
  extended by org.jruby.lexer.yacc.RubyYaccLexer

public class RubyYaccLexer
extends java.lang.Object

This is a port of the MRI lexer to Java it is compatible to Ruby 1.8.1.


Nested Class Summary
static class RubyYaccLexer.Keyword
           
static class RubyYaccLexer.LexState
           
 
Field Summary
static org.jcodings.Encoding ASCII8BIT_ENCODING
           
 boolean commandStart
           
static org.jcodings.Encoding USASCII_ENCODING
           
static org.jcodings.Encoding UTF8_ENCODING
           
 
Constructor Summary
RubyYaccLexer()
           
RubyYaccLexer(boolean isOneEight)
           
 
Method Summary
 boolean advance()
          How the parser advances to the next token.
 StrNode createStrNode(ISourcePosition position, org.jruby.util.ByteList buffer, int flags)
           
 StackState getCmdArgumentState()
           
 StackState getConditionState()
           
 java.lang.String getCurrentLine()
           
 org.jcodings.Encoding getEncoding()
           
static RubyYaccLexer.Keyword getKeyword(java.lang.String str)
           
 int getLeftParenBegin()
           
 ISourcePosition getPosition()
           
 ISourcePosition getPosition(ISourcePosition startPosition)
          Get position information for Token/Node that follows node represented by startPosition and current lexer location.
 StrTerm getStrTerm()
           
 java.lang.StringBuilder getTokenBuffer()
           
protected  void handleFileEncodingComment(org.jruby.util.ByteList encodingLine)
           
 int incrementParenNest()
           
 boolean isIdentifierChar(int c)
          This is a valid character for an identifier?
protected  boolean isMultiByteChar(int c)
          Is this a multibyte character from a multibyte encoding?
 boolean isOneEight()
           
 int nextToken()
           
protected  boolean parseMagicComment(org.jruby.util.ByteList magicLine)
           
protected  int readComment()
          Read a comment up to end of line.
 int readEscape()
           
 int readUTFEscape(org.jruby.util.ByteList buffer, boolean stringLiteral, boolean symbolLiteral)
           
 void readUTFEscapeRegexpLiteral(org.jruby.util.ByteList buffer)
           
 void reset()
           
 void resetStacks()
           
 void setEncoding(org.jcodings.Encoding encoding)
           
 void setLeftParenBegin(int value)
           
 void setParserSupport(ParserSupport parserSupport)
          Parse must pass its support object for some check at bottom of yylex().
 void setSource(LexerSource source)
          Allow the parser to set the source for its lexer.
 void setState(RubyYaccLexer.LexState state)
           
 void setStrTerm(StrTerm strterm)
           
 void setValue(java.lang.Object yaccValue)
           
 void setWarnings(IRubyWarnings warnings)
           
 int token()
          Last token read from the lexer at the end of a call to yylex()
 int tokenAddMBC(int codepoint, org.jruby.util.ByteList buffer)
           
 void tokenAddMBCFromSrc(int c, org.jruby.util.ByteList buffer)
           
 java.lang.Object value()
          Value of last token (if it is a token which has a value).
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

UTF8_ENCODING

public static final org.jcodings.Encoding UTF8_ENCODING

USASCII_ENCODING

public static final org.jcodings.Encoding USASCII_ENCODING

ASCII8BIT_ENCODING

public static final org.jcodings.Encoding ASCII8BIT_ENCODING

commandStart

public boolean commandStart
Constructor Detail

RubyYaccLexer

public RubyYaccLexer()

RubyYaccLexer

public RubyYaccLexer(boolean isOneEight)
Method Detail

getEncoding

public org.jcodings.Encoding getEncoding()

getKeyword

public static RubyYaccLexer.Keyword getKeyword(java.lang.String str)

incrementParenNest

public int incrementParenNest()

getLeftParenBegin

public int getLeftParenBegin()

setLeftParenBegin

public void setLeftParenBegin(int value)

reset

public final void reset()

advance

public boolean advance()
                throws java.io.IOException
How the parser advances to the next token.

Returns:
true if not at end of file (EOF).
Throws:
java.io.IOException

nextToken

public int nextToken()
              throws java.io.IOException
Throws:
java.io.IOException

token

public int token()
Last token read from the lexer at the end of a call to yylex()

Returns:
last token read

getTokenBuffer

public java.lang.StringBuilder getTokenBuffer()

value

public java.lang.Object value()
Value of last token (if it is a token which has a value).

Returns:
value of last value-laden token

getPosition

public ISourcePosition getPosition(ISourcePosition startPosition)
Get position information for Token/Node that follows node represented by startPosition and current lexer location.

Parameters:
startPosition - previous node/token
inclusive - include previous node into position information of current node
Returns:
a new position

getPosition

public ISourcePosition getPosition()

getCurrentLine

public java.lang.String getCurrentLine()

setParserSupport

public void setParserSupport(ParserSupport parserSupport)
Parse must pass its support object for some check at bottom of yylex(). Ruby does it this way as well (i.e. a little parsing logic in the lexer).

Parameters:
parserSupport -

setEncoding

public void setEncoding(org.jcodings.Encoding encoding)

setSource

public void setSource(LexerSource source)
Allow the parser to set the source for its lexer.

Parameters:
source - where the lexer gets raw data

getStrTerm

public StrTerm getStrTerm()

setStrTerm

public void setStrTerm(StrTerm strterm)

resetStacks

public void resetStacks()

setWarnings

public void setWarnings(IRubyWarnings warnings)

setState

public void setState(RubyYaccLexer.LexState state)

getCmdArgumentState

public StackState getCmdArgumentState()

isOneEight

public boolean isOneEight()

getConditionState

public StackState getConditionState()

setValue

public void setValue(java.lang.Object yaccValue)

isIdentifierChar

public boolean isIdentifierChar(int c)
This is a valid character for an identifier?

Parameters:
c - is character to be compared
Returns:
whether c is an identifier or not mri: is_identchar

isMultiByteChar

protected boolean isMultiByteChar(int c)
Is this a multibyte character from a multibyte encoding?

Parameters:
c -
Returns:
whether c is an multibyte char or not

createStrNode

public StrNode createStrNode(ISourcePosition position,
                             org.jruby.util.ByteList buffer,
                             int flags)

parseMagicComment

protected boolean parseMagicComment(org.jruby.util.ByteList magicLine)
                             throws java.io.IOException
Throws:
java.io.IOException

handleFileEncodingComment

protected void handleFileEncodingComment(org.jruby.util.ByteList encodingLine)
                                  throws java.io.IOException
Throws:
java.io.IOException

readComment

protected int readComment()
                   throws java.io.IOException
Read a comment up to end of line.

Returns:
something or eof value
Throws:
java.io.IOException

readUTFEscapeRegexpLiteral

public void readUTFEscapeRegexpLiteral(org.jruby.util.ByteList buffer)
                                throws java.io.IOException
Throws:
java.io.IOException

tokenAddMBC

public int tokenAddMBC(int codepoint,
                       org.jruby.util.ByteList buffer)

tokenAddMBCFromSrc

public void tokenAddMBCFromSrc(int c,
                               org.jruby.util.ByteList buffer)
                        throws java.io.IOException
Throws:
java.io.IOException

readUTFEscape

public int readUTFEscape(org.jruby.util.ByteList buffer,
                         boolean stringLiteral,
                         boolean symbolLiteral)
                  throws java.io.IOException
Throws:
java.io.IOException

readEscape

public int readEscape()
               throws java.io.IOException
Throws:
java.io.IOException


Copyright © 2002-2009 JRuby Team. All Rights Reserved.