class Syntax::Token

A single token extracted by a tokenizer. It is simply the lexeme itself, decorated with a 'group' attribute to identify the type of the lexeme.

Attributes

group[R]

the type of the lexeme that was extracted.

instruction[R]

the instruction associated with this token (:none, :region_open, or :region_close)

Public Class Methods

new( text, group, instruction = :none ) click to toggle source

Create a new Token representing the given text, and belonging to the given group.

Calls superclass method
# File lib/syntax/common.rb, line 19
def initialize( text, group, instruction = :none )
  super text
  @group = group
  @instruction = instruction
end