org.apache.manifoldcf.scriptengine

Class BasicTokenStream

java.lang.Object

org.apache.manifoldcf.scriptengine.BasicTokenStream

All Implemented Interfaces:

TokenStream

public class BasicTokenStream
extends java.lang.Object
implements TokenStream

Convert a sequence of characters into a stream of tokens.

Field Summary

Fields

Modifier and Type	Field and Description
protected CharacterStream	characterStream
protected Token	currentToken

Constructor Summary

Constructors

Constructor and Description
BasicTokenStream(java.io.Reader reader)

Method Summary

Modifier and Type	Method and Description
protected static boolean	isNumberCharacter(char x)
protected static boolean	isQuoteCharacter(char x)
protected static boolean	isTokenCharacter(char x)
protected Token	parseNextToken()
Token	peek() Examine the current token.
void	skip() Skip the current token.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

characterStream

protected CharacterStream characterStream

currentToken

protected Token currentToken

Constructor Detail

BasicTokenStream

public BasicTokenStream(java.io.Reader reader)

Method Detail

peek

public Token peek()
           throws ScriptException

Description copied from interface: TokenStream

Examine the current token.

Specified by:

peek in interface TokenStream

Throws:

ScriptException

skip

public void skip()

Description copied from interface: TokenStream

Skip the current token.

Specified by:

skip in interface TokenStream

parseNextToken

protected Token parseNextToken()
                        throws ScriptException

Throws:

ScriptException

isQuoteCharacter

protected static boolean isQuoteCharacter(char x)

isNumberCharacter

protected static boolean isNumberCharacter(char x)

isTokenCharacter

protected static boolean isTokenCharacter(char x)