Google Git
Sign in
android / platform / external / apache-velocity-engine / 02cf6db2 / . / src / java / org / apache / velocity / runtime / parser / Parser.jjt
blob: 105a060943c256d4c3e830947779cdadf6acddee [file] [log] [blame]
/*
* The Apache Software License, Version 1.1
*
* Copyright (c) 2000 The Apache Software Foundation. All rights
* reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
*
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in
* the documentation and/or other materials provided with the
* distribution.
*
* 3. The end-user documentation included with the redistribution, if
* any, must include the following acknowlegement:
* "This product includes software developed by the
* Apache Software Foundation (http://www.apache.org/)."
* Alternately, this acknowlegement may appear in the software itself,
* if and wherever such third-party acknowlegements normally appear.
*
* 4. The names "The Jakarta Project", "Tomcat", and "Apache Software
* Foundation" must not be used to endorse or promote products derived
* from this software without prior written permission. For written
* permission, please contact apache@apache.org.
*
* 5. Products derived from this software may not be called "Apache"
* nor may "Apache" appear in their names without prior written
* permission of the Apache Group.
*
* THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
* WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
* DISCLAIMED. IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR
* ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
* USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
* ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
* OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
* OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
* ====================================================================
*
* This software consists of voluntary contributions made by many
* individuals on behalf of the Apache Software Foundation. For more
* information on the Apache Software Foundation, please see
* <http://www.apache.org/>.
*/
/*
* NOTE : please see documentation at bottom of this file. (It was placed there its tiring
* to always have to page past it... :)
*/
options
{
/** The default package for this parser kit */
NODE_PACKAGE="org.apache.velocity.runtime.parser";
/** A source file will be generated for each non-terminal */
MULTI=true;
/**
* Each node will have access to the parser, I did this so
* some global information can be shared via the parser. I
* think this will come in handly keeping track of
* context, and being able to push changes back into
* the context when nodes make modifications to the
* context by setting properties, variables and
* what not.
*/
NODE_USES_PARSER=true;
/**
* The parser must be non-static in order for the
* above option to work, otherwise the parser value
* is passed in as null, which isn't all the useful ;)
*/
STATIC=false;
/**
* Enables the use of a visitor that each of nodes
* will accept. This way we can separate the logic
* of node processing in a visitor and out of the
* nodes themselves. If processing changes then
* the nothing has to change in the node code.
*/
VISITOR=true;
/**
* This option is used as one of the steps
* required to allow the use of an "#include"
* type behaviour. In this case the directive
* is "#parse". See the TOKEN_MGR_DECLS section
* below for details on how the TokenManager is
* modified to allow this behaviour.
*/
COMMON_TOKEN_ACTION=true;
DEBUG_PARSER=false;
DEBUG_TOKEN_MANAGER=false;
}
PARSER_BEGIN(Parser)
package org.apache.velocity.runtime.parser;
import java.io.*;
import java.util.*;
import org.apache.velocity.runtime.Runtime;
import org.apache.velocity.runtime.parser.node.*;
import org.apache.velocity.runtime.directive.Directive;
import org.apache.velocity.runtime.directive.Macro;
import org.apache.velocity.util.StringUtils;
/**
* This class is responsible for parsing a Velocity
* template. This class was generated by JavaCC using
* the JJTree extension to produce an Abstract
* Syntax Tree (AST) of the template.
*
* Please look at the Parser.jjt file which is
* what controls the generation of this class.
*
* @author <a href="mailto:jvanzyl@periapt.com">Jason van Zyl</a>
* @author <a href="mailto:geirm@optonline.net">Geir Magnusson Jr.</a>
* @version $Id: Parser.jjt,v 1.33 2000/11/22 02:17:25 geirm Exp $
*/
public class Parser
{
/**
* This Hashtable contains a list of all of the dynamic directives.
*/
private Hashtable directives = new Hashtable(0);
/** This was added to allow the parser to be associated
* with a particular syntax. JavaCC doesn't generate
* a constructor without parameters. The normal constructor
* takes a single argument which an InputStream. But in
* order to make the parser dynamically loadable this
* constructor had to be added. This also allows us to
* create a single instance of a parser and reuse
* it over and over.
*/
public Parser()
{
this(new ByteArrayInputStream("\n".getBytes()));
}
/** This was also added to allow parsers to be dynamically
* loadable.
*
* Taken from the generated constructor in Parser.java.
* Just be watchful when you change the grammar because
* the generated method changes when the grammar changes
* WRT to adding new token types. So you have to
* occasionally do some cutting and pasting :-)
*
* It would be A LOT better it you could subclass grammars
* and override particular methods but that's not
* possible with JavaCC. I believe that you can do
* this with ANTLR though.
*/
public SimpleNode parse(InputStream stream) throws ParseException
{
SimpleNode sn = null;
try
{
token_source.clearStateVars();
ReInit(stream);
sn = process();
}
catch (ParseException pe)
{
Runtime.error ("Parser Exception: " + StringUtils.stackTrace(pe));
throw new ParseException (pe.currentToken,
pe.expectedTokenSequences, pe.tokenImage);
}
catch (Exception e)
{
Runtime.error ("Parser Error: " + StringUtils.stackTrace(e));
}
return sn;
}
/**
* This method sets the directives Hashtable
*/
public void setDirectives(Hashtable directives)
{
this.directives = directives;
}
/**
* This method gets the directives Hashtable
*/
public Directive getDirective(String directive)
{
return (Directive) directives.get(directive);
}
/**
* This method finds out of the directive exists in the directives
* Hashtable.
*/
public boolean isDirective(String directive)
{
if (directives.containsKey(directive))
return true;
else
return false;
}
/**
* Produces a processed output for an escaped control or pluggable directive
*/
private String escapedDirective( String strImage )
{
int iLast = strImage.lastIndexOf("\\");
String strDirective = strImage.substring(iLast + 1);
boolean bRecognizedDirective = false;
/*
* is this a PD or a control directive?
*/
if ( isDirective( strDirective.substring(1)))
{
bRecognizedDirective = true;
}
else if (Runtime.isVelocimacro( strDirective.substring(1)))
{
bRecognizedDirective = true;
}
else
{
/* order for speed? */
if ( strDirective.substring(1).equals("if")
|| strDirective.substring(1).equals("end")
|| strDirective.substring(1).equals("set")
|| strDirective.substring(1).equals("else")
|| strDirective.substring(1).equals("elseif")
|| strDirective.substring(1).equals("stop")
)
{
bRecognizedDirective = true;
}
}
/*
* if so, make the proper prefix string (let the escapes do their thing..)
* otherwise, just return what it is..
*/
if (bRecognizedDirective)
return ( strImage.substring(0,iLast/2) + strDirective);
else
return ( strImage );
}
}
PARSER_END(Parser)
/**
* This gets inserted into the ParserMacroTokenManager
* and is being used here strictly for the #parse
* directive: an #include type behaviour. We have
* to save the state the stream currently being
* parsed and we have to save the state of the
* lexer (TokenManager class) then we create
* a new stream from the file named in the
* #parse directive then we ReInit the lexer.
* Whatever it parses will get placed
* into the AST.
*
* I need a simple way to detect circular
* inclusions so this thing doesn't go wild
* and drag down the VM.
*/
TOKEN_MGR_DECLS:
{
private int fileDepth = 0;
private int lparen = 0;
private int rparen = 0;
Stack stateStack = new Stack();
public boolean bDebugPrint_ = false;
private boolean inReference;
public boolean inDirective;
private boolean inComment;
private boolean inSet;
Stack streams = new Stack();
Stack states = new Stack();
/**
* Retrieve the oldStream and oldState and
* continue processing the input.
*/
void popFile()
{
ReInit((ASCII_CharStream) streams.pop(), ((Integer) states.pop()).intValue());
fileDepth--;
}
private boolean AtParent()
{
if (fileDepth == 0)
return true;
else
return false;
}
void CommonTokenAction(Token t)
{
if (t.kind == EOF && ! AtParent())
{
Token new_t;
popFile();
new_t = getNextToken();
t.kind = new_t.kind;
t.beginLine = new_t.beginLine;
t.beginColumn = new_t.beginColumn;
t.endLine = new_t.endLine;
t.endColumn = new_t.endColumn;
t.image = new_t.image;
t.next = new_t.next;
t.specialToken = new_t.specialToken;
}
}
/**
* pushes the current state onto the 'state stack',
* and maintains the parens counts
* public because we need it in PD & VM handling
*
* @return boolean : success. It can fail if the state machine
* gets messed up.
*/
public boolean stateStackPop()
{
Hashtable hStack;
try
{
hStack = (Hashtable) stateStack.pop();
}
catch( EmptyStackException e)
{
lparen=0;
SwitchTo(DEFAULT);
return false;
}
if( bDebugPrint_ )
System.out.println(" stack pop (" + stateStack.size() + ") : lparen=" + ( (Integer) hStack.get("lparen")).intValue()
+ " newstate=" + ( (Integer) hStack.get("lexstate")).intValue() );
lparen = ( (Integer) hStack.get("lparen")).intValue();
rparen = ( (Integer) hStack.get("rparen")).intValue();
SwitchTo( ( (Integer) hStack.get("lexstate")).intValue() );
return true;
}
/**
* pops a state off the stack, and restores paren counts
*
* @return boolean : success of operation
*/
public boolean stateStackPush()
{
if( bDebugPrint_ )
System.out.println(" (" + stateStack.size() + ") pushing cur state : " + curLexState );
Hashtable hStack = new Hashtable();
hStack.put("lexstate", new Integer( curLexState ) );
hStack.put("lparen", new Integer( lparen ));
hStack.put("rparen", new Integer( rparen ));
lparen = 0;
stateStack.push( hStack );
return true;
}
/**
* Clears all state variables, resets to
* start values, clears stateStack. Call
* before parsing.
* @return void
*/
public void clearStateVars()
{
stateStack.clear();
lparen = 0;
rparen = 0;
inReference = false;
inDirective = false;
inComment = false;
inSet = false;
return;
}
/**
* handles the dropdown logic when encountering a RPAREN
*/
private void RPARENHandler()
{
/*
* Ultimately, we want to drop down to the state below the one that has an open (
* if we hit bottom (DEFAULT), that's fine. It's just text schmoo.
*
*/
boolean bClosed = false;
if (inComment)
bClosed = true;
while( !bClosed)
{
/*
* look at current state. If we haven't seen a lparen in this state
* then we drop a state, because this lparen clearly closes our state
*/
if( lparen > 0)
{
/*
* if rparen + 1 == lparen, then this state is closed. Otherwise, increment
* and keep parsing
*/
if( lparen == rparen + 1)
{
stateStackPop();
}
else
{
rparen++;
}
bClosed = true;
}
else
{
/*
* now, drop a state
*/
if(!stateStackPop())
break;
}
}
}
}
/* ------------------------------------------------------------------------
*
* Tokens
*
* Note : we now have another state, REFMODIFIER. This is sort of a
* type of REFERENCE state, simply use to use the DIRECTIVE token
* set when we are processing a $foo.bar() construct
*
* ------------------------------------------------------------------------- */
<DIRECTIVE,REFMODIFIER,REFMOD2>
TOKEN:
{
<LBRACKET: "[">
| <RBRACKET: "]">
}
<DIRECTIVE,REFMOD2>
TOKEN:
{
<COMMA:",">
}
<DIRECTIVE,REFMODIFIER>
TOKEN:
{
<LPAREN: "(">
{
if (!inComment)
lparen++;
/*
* if in REFERENCE and we have seen the dot, then move to REFMOD2 -> Modifier()
*/
if (curLexState == REFMODIFIER )
SwitchTo( REFMOD2 );
}
}
/*
* we never will see a ')' in anything but DIRECTIVE and REFMOD2. Each have their own
*/
<DIRECTIVE>
TOKEN:
{
/*
* We will eat any whitespace upto and including a newline for directives
*/
<RPAREN: ")" ( ( " " | "\t" )* ( "\n" | "\r" | "\r\n" ))?>
{
RPARENHandler();
}
}
<REFMOD2>
TOKEN:
{
/*
* in REFMOD2, we don't want to bind the whitespace and \n like we do when closing a directive.
*/
<REFMOD2_RPAREN: ")">
{
/*
* need to simply switch back to REFERENCE, not drop down the stack
* because we can (infinitely) chain, ala $foo.bar().blargh().woogie().doogie()
*/
SwitchTo( REFERENCE );
}
}
/*----------------------------------------------
*
* escape "\\" handling for the built-in directives
*
*--------------------------------------------- */
TOKEN:
{
/*
* We have to do this, because we want these to be a Text node, and
* whatever follows to be peer to this text in the tree.
*
* We need to touch the ASTs for these, because we want an even # of \'s
* to render properly in front of the block
*
* This is really simplistic. I actually would prefer to find them in
* grammatical context, but I am neither smart nor rested, a receipe
* for disaster, another long night with Mr. Parser, or both.
*/
<ESCAPE_DIRECTIVE : (<DOUBLE_ESCAPE>)* "\\#" <WORD> >
}
/*
* needed because #set is so wacky in it's desired behavior. We want set to eat any preceeding whitespace
* so it is invisible in formatting. (As it should be.) If this works well, I am going to chuck the whole MORE:
* token abomination.
*/
TOKEN:
{
<SET_DIRECTIVE: (" "|"\t")* "#set" >
{
if (! inComment)
{
inDirective = true;
if ( bDebugPrint_ )
System.out.print("#set : going to " + DIRECTIVE );
stateStackPush();
inSet = true;
SwitchTo(DIRECTIVE);
}
}
}
<*>
MORE :
{
/*
* Note : DOLLARBANG is a duplicate of DOLLAR. They must be identical.
*/
<DOLLAR: ("\\")* "$">
{
if (! inComment)
{
inReference = true;
if ( bDebugPrint_ )
System.out.print( "$ : going to " + REFERENCE );
stateStackPush();
SwitchTo(REFERENCE);
}
}
| <DOLLARBANG: "$!">
{
if (! inComment)
{
inReference = true;
if ( bDebugPrint_ )
System.out.print( "$! : going to " + REFERENCE );
stateStackPush();
SwitchTo(REFERENCE);
}
}
| "##"
{
if (!inComment)
{
inComment = true;
stateStackPush();
SwitchTo(IN_SINGLE_LINE_COMMENT);
}
}
| <"#**" ~["#"]>
{
input_stream.backup(1);
inComment = true;
stateStackPush();
SwitchTo( IN_FORMAL_COMMENT);
}
| "#*"
{
inComment=true;
stateStackPush();
SwitchTo( IN_MULTI_LINE_COMMENT );
}
| <HASH : "#" >
{
if (! inComment)
{
/*
* We can have the situation where #if($foo)$foo#end. We need to transition out of
* REFERENCE before going to DIRECTIVE. I don't really like this, but I can't think of
* a legal way you are going into DIRECTIVE while in REFERENCE. -gmj
*/
if (curLexState == REFERENCE)
{
inReference = false;
stateStackPop();
}
inDirective = true;
if ( bDebugPrint_ )
System.out.print("# : going to " + DIRECTIVE );
stateStackPush();
SwitchTo(PRE_DIRECTIVE);
}
}
}
TOKEN :
{
<DOUBLE_ESCAPE : "\\\\">
| <ESCAPE: "\\" >
| <TEXT: (~["$", "#", "\\"])+ >
}
/* -----------------------------------------------------------------------
*
* *_COMMENT Lexical tokens
*
*-----------------------------------------------------------------------*/
<IN_SINGLE_LINE_COMMENT>
TOKEN :
{
<SINGLE_LINE_COMMENT: "\n" | "\r" | "\r\n" >
{
inComment = false;
stateStackPop();
}
}
<IN_FORMAL_COMMENT>
TOKEN :
{
<FORMAL_COMMENT: "*#" >
{
inComment = false;
stateStackPop();
}
}
<IN_MULTI_LINE_COMMENT>
TOKEN :
{
<MULTI_LINE_COMMENT: "*#" >
{
inComment = false;
stateStackPop();
}
}
<IN_SINGLE_LINE_COMMENT,IN_FORMAL_COMMENT,IN_MULTI_LINE_COMMENT>
MORE :
{
< ~[] >
}
/* -----------------------------------------------------------------------
*
* DIRECTIVE Lexical State (some of it, anyway)
*
* ---------------------------------------------------------------------- */
<DIRECTIVE,REFMOD2>
TOKEN:
{
<WHITESPACE : ([" ","\t"])+ >
}
<REFERENCE,DIRECTIVE,REFMODIFIER,REFMOD2>
TOKEN :
{
<STRING_LITERAL: ( "\"" ( ~["\"","\n","\r"] )* "\"" ) >
{
/*
* - if we are in REFERENCE || REFMODIFIER then " is an ender
* - if we are in DIRECTIVE and haven't seen ( yet, then also drop out.
* don't forget to account for the beloved yet wierd #set
* - finally, if we are in REFMOD2 (remember : $foo.bar( ) then " is ok!
*/
if (curLexState == REFERENCE || curLexState == REFMODIFIER)
stateStackPop();
else if( curLexState == DIRECTIVE && !inSet && lparen == 0)
stateStackPop();
}
| <TRUE: "true">
| <FALSE: "false">
}
<DIRECTIVE>
TOKEN :
{
<NEWLINE: "\n" | "\r" | "\r\n" >
{
if ( bDebugPrint_ )
System.out.println(" NEWLINE :");
stateStackPop();
if (inSet)
inSet = false;
if (inDirective)
inDirective = false;
}
}
<DIRECTIVE>
TOKEN :
{
<MINUS: "-">
| <PLUS: "+">
| <MULTIPLY: "*">
| <DIVIDE: "/">
| <MODULUS: "%">
| <LOGICAL_AND: "&&">
| <LOGICAL_OR: "||">
| <LOGICAL_LT: "<">
| <LOGICAL_LE: "<=">
| <LOGICAL_GT: ">">
| <LOGICAL_GE: ">=">
| <LOGICAL_EQUALS: "==">
| <LOGICAL_NOT_EQUALS: "!=">
| <LOGICAL_NOT: "!">
| <EQUALS: "=" >
}
<PRE_DIRECTIVE>
TOKEN :
{
<END: "end" ( ( " " | "\t" )* ( "\n" | "\r" | "\r\n" ) )? >
{
inDirective = false;
stateStackPop();
}
| <IF_DIRECTIVE: "if">
{
SwitchTo(DIRECTIVE);
}
| <ELSEIF_DIRECTIVE: "elseif">
{
SwitchTo(DIRECTIVE);
}
| <ELSE_DIRECTIVE: "else">
{
inDirective = false;
stateStackPop();
}
| <STOP_DIRECTIVE: "stop">
{
matchedToken.kind = EOF;
fileDepth = 0;
}
}
<PRE_DIRECTIVE,DIRECTIVE,REFMOD2>
TOKEN:
{
<#DIGIT: [ "0"-"9" ] >
| <NUMBER_LITERAL: (<DIGIT>)+ >
{
/*
* check to see if we are in set
* ex. #set $foo = $foo + 3
* because we want to handle the \n after
*/
if ( lparen == 0 && !inSet && curLexState != REFMOD2)
stateStackPop();
}
}
<PRE_DIRECTIVE,DIRECTIVE>
TOKEN:
{
<#LETTER: [ "a"-"z", "A" - "Z" ] >
| <WORD: (<LETTER>)+ >
}
/* -----------------------------------------------------------------------
*
* REFERENCE Lexical States
*
* This is more than a single state, because of the structure of
* the VTL references. We use three states because the set of tokens
* for each state can be different.
*
* $foo.bar( "arg" )
* ^ ^ ^
* | | |
* ----------- > REFERENCE : state initiated by the '$' character. Continues
* | | until end of the reference, or the . character.
* |------ > REFMODIFIER : state switched to when the <DOT> is encountered.
* | note that this is a switch, not a push. See notes at bottom
* | re stateStack.
* |-- > REFMOD2 : state switch to when the LPAREN is encountered.
* again, this is a switch, not a push.
*
* ---------------------------------------------------------------------------- */
<REFERENCE,REFMODIFIER,REFMOD2>
TOKEN :
{
<#ALPHA_CHAR: ["a"-"z", "A"-"Z"] >
| <#ALPHANUM_CHAR: [ "a"-"z", "A"-"Z", "0"-"9" ] >
| <#IDENTIFIER_CHAR: [ "a"-"z", "A"-"Z", "0"-"9", "-", "_" ] >
| <IDENTIFIER: <ALPHA_CHAR> (<IDENTIFIER_CHAR>)* >
| <DOT: "." <ALPHA_CHAR>>
{
/*
* push the alpha char back into the stream so the following identifier is complete
*/
input_stream.backup(1);
/*
* and munge the <DOT> so we just get a . when we have normal text that looks like a ref.ident
*/
matchedToken.image = ".";
if ( bDebugPrint_ )
System.out.print("DOT : switching to " + REFMODIFIER);
SwitchTo(REFMODIFIER);
}
| <LCURLY: "{">
| <RCURLY: "}">
{
stateStackPop();
}
}
<REFERENCE,REFMODIFIER,REFMOD2>
SPECIAL_TOKEN :
{
<REFERENCE_TERMINATOR: ~[] >
{
/*
* push every terminator character back into the stream
*/
input_stream.backup(1);
inReference = false;
if ( bDebugPrint_ )
System.out.print("REF_TERM :");
stateStackPop();
}
}
<PRE_DIRECTIVE>
SPECIAL_TOKEN :
{
<DIRECTIVE_TERMINATOR: ~[] >
{
if ( bDebugPrint_ )
System.out.print("DIRECTIVE_TERM :");
input_stream.backup(1);
inDirective = false;
stateStackPop();
}
}
/**
* This method is what starts the whole parsing
* process. After the parsing is complete and
* the template has been turned into an AST,
* this method returns the root of AST which
* can subsequently be traversed by a visitor
* which implements the ParserVisitor interface
* which is generated automatically by JavaCC
*/
SimpleNode process() : {}
{
( Statement() )* <EOF>
{ return jjtThis; }
}
/**
* These are the types of statements that
* are acceptable in Velocity templates.
*/
void Statement() #void : {}
{
IfStatement()
| StopStatement()
| Reference()
| Comment()
| SetDirective()
| EscapedDirective()
| Escape()
| Directive()
| Text()
}
/**
* used to separate the notion of a valid directive that has been
* escaped, versus something that looks like a directive and
* is just schmoo. This is important to do as a separate production
* that creates a node, because we want this, in either case, to stop
* the further parsing of the Directive() tree.
*/
void EscapedDirective() : {}
{
{
Token t = null;
}
t = <ESCAPE_DIRECTIVE>
{
/*
* churn and burn..
*/
t.image = escapedDirective( t.image );
}
}
/**
* Used to catch and process escape sequences in grammatical constructs
* as escapes outside of VTL are just characters. Right now we hav both
* this and the EscapeDirective() construction because in the EscapeDirective()
* case, we want to suck in the #<directive> and here we don't. We just want
* the escapes to render correctly
*/
void Escape() : {}
{
{
Token t = null;
int iCount = 0;
boolean bControl = false;
}
( LOOKAHEAD(2) t = <DOUBLE_ESCAPE>
{
iCount++;
}
)+
{
/*
* first, check to see if we have a control directive
*/
switch(t.next.kind ) {
case IF_DIRECTIVE :
case ELSE_DIRECTIVE :
case ELSEIF_DIRECTIVE :
case END :
case STOP_DIRECTIVE :
bControl = true;
break;
}
/*
* if that failed, lets lookahead to see if we matched a PD or a VM
*/
if ( isDirective( t.next.image.substring(1)))
bControl = true;
else if ( Runtime.isVelocimacro( t.next.image.substring(1)))
bControl = true;
t.image = "";
for( int i = 0; i < iCount; i++)
t.image += (bControl ? "\\" : "\\\\");
}
}
void Comment() : {}
{
<SINGLE_LINE_COMMENT>
| <MULTI_LINE_COMMENT>
| <FORMAL_COMMENT>
}
void NumberLiteral() : {}
{
<NUMBER_LITERAL>
}
void StringLiteral() : {}
{
<STRING_LITERAL>
}
/**
* This method corresponds to variable
* references in Velocity templates.
* The following are examples of variable
* references that may be found in a
* template:
*
* $foo
* $bar
*
*/
void Identifier() : {}
{
<IDENTIFIER>
}
void Word() : {}
{
<WORD>
}
/**
* Supports the arguments for the Pluggable Directives
* We add whitespace in here as a token so the VMs can
* easily reconstruct a macro body from the token stream
* @see Directive()
*/
void DirectiveArg() #void : {}
{
Reference()
| Word()
| StringLiteral()
| NumberLiteral()
| ObjectArray()
| <WHITESPACE>
}
/**
* Supports the Pluggable Directives
* #foo( arg+ )
*/
SimpleNode Directive() :
{
Token t = null;
Directive d;
boolean bDoItNow = false;
}
{
/*
* note that if we were escaped, that is now handled by EscapedDirective()
*/
t = <WORD>
{
String strDirectiveName = t.image.substring(1);
d = (Directive) directives.get( strDirectiveName );
/*
* Velocimacro support : if the directive is macro directive
* then set the flag so after the block parsing, we add the VM
* right then. (So available if used w/in the current template )
*/
if ( strDirectiveName.equals("macro"))
{
bDoItNow = true;
}
/*
* set the directive name from here. No reason for the thing to know about parser tokens
*/
jjtThis.setDirectiveName( strDirectiveName );
if ( d == null)
{
/*
* if null, then not a real directive, but maybe a Velocimacro
*/
d = (Directive) Runtime.getVelocimacro( strDirectiveName );
if (d == null)
{
token_source.stateStackPop();
token_source.inDirective = false;
return jjtThis;
}
}
/*
* now, switch us out of PRE_DIRECTIVE
*/
token_source.SwitchTo(DIRECTIVE);
}
/*
* if this is indeed a token, match the #foo ( arg ) pattern
*/
[<WHITESPACE>] <LPAREN> ( DirectiveArg() )* <RPAREN>
{
if (d.getType() == Directive.LINE)
return jjtThis;
}
/*
* and the following block if the PD needs it
*/
( Statement() )+ #Block
<END>
{
/*
* VM : if we are processing a #macro directive, we need to
* process the block. In truth, I can just register the name
* and do the work later when init-ing. That would work
* as long as things were always defined before use. This way
* we don't have to worry about forward references and such...
*/
if (bDoItNow)
{
Macro m = new Macro();
m.processAndRegister( jjtThis );
}
/*
* VM : end
*/
return jjtThis;
}
}
void ObjectArray() : {}
{
<LBRACKET> [ Parameter() ( <COMMA> Parameter() )* ] <RBRACKET>
}
/**
* This method has yet to be fully implemented
* but will allow arbitrarily nested method
* calls
*/
void Parameter() #void: {}
{
[<WHITESPACE>]
(
StringLiteral()
| ObjectArray()
| True()
| False()
| Reference()
| NumberLiteral()
)
[<WHITESPACE>]
}
/**
* This method has yet to be fully implemented
* but will allow arbitrarily nested method
* calls
*/
void Method() : {}
{
Identifier() <LPAREN> [ Parameter() ( <COMMA> Parameter() )* ] <REFMOD2_RPAREN>
}
void Reference() : {}
{
// This should be changed to Indentifier() now. Make
// it easier to walk the AST.
[<LCURLY>]
<IDENTIFIER>
[<RCURLY>]
(LOOKAHEAD(2) <DOT> (LOOKAHEAD(3) Method() | Identifier() ) [<RCURLY>] )*
}
void True() : {}
{
<TRUE>
}
void False() : {}
{
<FALSE>
}
/**
* This method is responsible for allowing
* all non-grammar text to pass through
* unscathed.
*/
void Text() : {}
{
<TEXT>
| <DOT>
| <RPAREN>
| <LPAREN>
| <NUMBER_LITERAL>
| <STRING_LITERAL>
| <ESCAPE>
}
/* -----------------------------------------------------------------------
*
* Defined Directive Syntax
*
* ----------------------------------------------------------------------*/
void IfStatement() : {}
{
<IF_DIRECTIVE> [<WHITESPACE>] <LPAREN> Expression() <RPAREN>
( Statement() )+ #Block
[ LOOKAHEAD(1) ( ElseIfStatement() )+ ]
[ LOOKAHEAD(1) ElseStatement() ]
<END>
}
void ElseStatement() : {}
{
<ELSE_DIRECTIVE>
( Statement() )+ #Block
}
void ElseIfStatement() : {}
{
<ELSEIF_DIRECTIVE> [<WHITESPACE>]
<LPAREN> Expression() <RPAREN>
( Statement() )+ #Block
}
void SetDirective() : {}
{
( <SET_DIRECTIVE> Expression() [<NEWLINE>] )
}
/**
* This method corresponds to the #stop
* directive which just simulates and EOF
* so that parsing stops. The #stop directive
* is useful for end-user debugging
* purposes.
*/
void StopStatement() #void: {}
{
<STOP_DIRECTIVE>
}
/* -----------------------------------------------------------------------
*
* Expression Syntax
*
* ----------------------------------------------------------------------*/
void Expression() : {}
{
LOOKAHEAD( PrimaryExpression() <EQUALS> ) Assignment()
| ConditionalOrExpression()
}
void Assignment() #Assignment(2) : {}
{
PrimaryExpression() <EQUALS> Expression()
}
void ConditionalOrExpression() #void : {}
{
ConditionalAndExpression()
( <LOGICAL_OR> ConditionalAndExpression() #OrNode(2) )*
}
void ConditionalAndExpression() #void : {}
{
EqualityExpression()
( <LOGICAL_AND> EqualityExpression() #AndNode(2) )*
}
void EqualityExpression() #void : {}
{
RelationalExpression()
(
<LOGICAL_EQUALS> RelationalExpression() #EQNode(2)
| <LOGICAL_NOT_EQUALS> RelationalExpression() #NENode(2)
)*
}
void RelationalExpression() #void : {}
{
AdditiveExpression()
(
<LOGICAL_LT> AdditiveExpression() #LTNode(2)
| <LOGICAL_GT> AdditiveExpression() #GTNode(2)
| <LOGICAL_LE> AdditiveExpression() #LENode(2)
| <LOGICAL_GE> AdditiveExpression() #GENode(2)
)*
}
void AdditiveExpression() #void : {}
{
MultiplicativeExpression()
(
<PLUS> MultiplicativeExpression() #AddNode(2)
| <MINUS> MultiplicativeExpression() #SubtractNode(2)
)*
}
void MultiplicativeExpression() #void : {}
{
UnaryExpression()
(
<MULTIPLY> UnaryExpression() #MulNode(2)
| <DIVIDE> UnaryExpression() #DivNode(2)
| <MODULUS> UnaryExpression() #ModNode(2)
)*
}
void UnaryExpression() #void : {}
{
<LOGICAL_NOT> UnaryExpression() #NotNode(1)
| PrimaryExpression()
}
void PrimaryExpression() #void : {}
{
[<WHITESPACE>]
(
StringLiteral()
| NumberLiteral()
| Reference()
| ObjectArray()
| True()
| False()
| <LPAREN> Expression() <RPAREN>
)
[<WHITESPACE>]
}
/* ======================================================================
Notes
-----
template == the input stream for this parser, contains 'VTL' mixed in with 'schmoo'
VTL == Velocity Template Language : the references, directives, etc
shmoo == the non-VTL component of a template
reference == VTL entity that represents data within the context. ex. $foo
directive == VTL entity that denotes 'action' (#set, #foreach, #if )
defined directive (DD) == VTL directive entity that is expressed explicitly w/in this grammar
pluggable directive (PD) == VTL directive entity that is defined outside of the grammar. PD's
allow VTL to be easily expandable w/o parser modification.
The problem with parsing VTL is that an input stream consists generally of little
bits of VTL mixed in with 'other stuff, referred to as 'schmoo'. Unlike
other languages, like C or Java, where the parser can punt whenever it encounters
input that doesn't conform to the grammar, the VTL parser can't do that. It must simply
output the schmoo and keep going.
There are a few things that we do here :
- define a set of parser states (DEFAULT, DIRECTIVE, REFERENCE, etc)
- define for each parser state a set of tokens for each state
- define the VTL grammar, expressed (mostly) in the productions such as Text(), SetStatement(), etc.
It is clear that this expression of the VTL grammer (the contents of this .jjt file) is maturing and
evolving as we learn more about how to parse VTL ( and as I learn about parsing...), so in the event
this documentation is in disagreement w/ the source, the source takes precedence. :)
Parser States
-------------
DEFAULT : This is the base or starting state, and strangely enough, the default state.
PRE_DIRECTIVE : State immediately following '#' before we figure out which defined or pluggable
directive (or neither) we are working with.
DIRECTIVE : This state is triggered by the a match of a DD or a PD.
REFERENCE : Triggered by '$'. Analagous to PRE_DIRECTIVE.
REFMODIFIER : Triggered by .<alpha> when in REFERENCE.
REFMOD2 : Triggered by ( when in REFMODIFIER
(cont)
Escape Sequences
----------------
The escape processing in VTL is very simple. The '\' character acts only as an escape when :
1) On or more touch a VTL element.
A VTL element is either :
1) It preceeds a reference that is in the context.
2) It preceeds a defined directive (#set, #if, #end, etc) or a valid pluggable directive,
such as #foreach
In all other cases the '\' is just another piece of text. The purpose of this is to allow the non-VTL
parts of a template (the 'schmoo') to not have to be altered for processing by Velocity.
So if in the context $foo and $bar were defined and $woogie was not
\$foo \$bar \$woogie
would output
$foo $bar \$woogie
Further, you can stack them and they affect left to right, just like convention escape characters in other languages.
\$foo = $foo
\\$foo = \<foo>
\\\$foo = \$foo
What You Expect
---------------
The recent versions of the parser are trying to support precise output to support general template use.
The directives do not render trailing whitespace and newlines if followed by a newline. They will render preceeding whitespace.
The only exception is #set, which also eats preceeding whitespace.
So, with a template :
------
#set $foo="foo"
#if($foo)
\$foo = $foo
#end
------
it will render precisely :
------
$foo = foo
------
*/