|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
A Lexer reads input bytes (InputStream) or characters (Reader), until one of its terminals is fulfilled. This happens when the Parser calls getNextToken(). The terminals will be set by the Parser on init.
Usage:
SyntaxSeparation separation = new SyntaxSeparation(new Syntax(myRules)); LexerBuilder builder = new LexerBuilder(separation.getLexerSyntax(), separation.getIgnoredSymbols()); Lexer lexer = builder.getLexer(inputStream); lexer.setTerminals(separation.getTokenSymbols()); Token token; do { token = lexer.getNextToken(null); System.err.println("token.symbol="+token.symbol+", text >"+token.text+"<"); } while (token.symbol != null && Token.isEpsilon(token) == false); boolean error = token.symbol == null;
LexerImpl
Nested Class Summary | |
static interface |
Lexer.TokenListener
A way to receive every parsing syntax Token the Lexer reads, even it is ignored. |
Method Summary | |
void |
addTokenListener(Lexer.TokenListener listener)
Installs a TokenListener that wants to know about every read Token, even it is ignored. |
void |
clear()
Reset the Lexer for another pass. |
void |
dump(java.io.PrintStream out)
Dump the current text and the scan position. |
Token |
getNextToken(java.util.Map tokenSymbols)
Returns the next token from input. |
void |
removeTokenListener(Lexer.TokenListener listener)
Removes a TokenListener from this Lexer. |
void |
setDebug(boolean debug)
Turn on and off debug mode. |
void |
setInput(java.lang.Object text)
Set the input to be processed by the Lexer. |
void |
setTerminals(java.util.List terminals)
Tells the Lexer the terminals (tokens) to scan, called on init. |
Method Detail |
public void setInput(java.lang.Object text) throws java.io.IOException
text
- can be String, StringBuffer, File, InputStream, Reader.
java.io.IOException
public void setTerminals(java.util.List terminals)
terminals
- List of String containing all terminals of the parser syntax.public Token getNextToken(java.util.Map tokenSymbols) throws java.io.IOException
tokenSymbols
- a Map that contains token symbols (in "key" field) expected by the Parser, can be null (slower).
java.io.IOException
public void clear()
public void addTokenListener(Lexer.TokenListener listener)
listener
- the Lexer.Listener implementation to install.public void removeTokenListener(Lexer.TokenListener listener)
listener
- the Lexer.Listener implementation to remove.public void dump(java.io.PrintStream out)
public void setDebug(boolean debug)
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |