antlr-runtime/antlr-3.4/BUILD.txt

 [The "BSD license"]
 Copyright (c) 2010 Terence Parr
 Maven Plugin - Copyright (c) 2009      Jim Idle

 All rights reserved.

 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions
 are met:
 1. Redistributions of source code must retain the above copyright
    notice, this list of conditions and the following disclaimer.
 2. Redistributions in binary form must reproduce the above copyright
    notice, this list of conditions and the following disclaimer in the
    documentation and/or other materials provided with the distribution.
 3. The name of the author may not be used to endorse or promote products
    derived from this software without specific prior written permission.

 THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
 IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
 OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
 IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
 INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
 NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
 THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

============================================================================

This file contains the build instructions for the ANTLR toolset as
of version 3.1.3 and beyond.

The ANTLR toolset must be built using the Maven build system as
this build system updates the version numbers and controls the
whole build process. However, if you just want the latest build
and do not care to learn anything about Maven, then visit the 'target'
directories (for jars) under the depot mirror root here:

   http://antlr.org/depot

If you are looking for the latest released version of ANTLR, then
visit the downloads page on the main antlr.org website.

These instructions are mainly for the ANTLR development team,
though you are free to build ANTLR yourself of course.

Source code Structure
-----------------------

The main development branch of ANTLR is stored within the Perforce SCM at:

   //depot/code/antlr/main/...

release branches are stored in Perforce like so:

   //depot/code/antlr/release-3.1.3/...

In this top level directory, you will find a master build file for
Maven called pom.xml and you will also note that there are a number of
subdirectories:

 tool                  - The ANTLR tool itself
 runtime/Java          - The ANTLR Java runtime
 runtime/X             - The runtime for language target X
 gunit                 - The grammar test tool
 antlr3-maven-plugin   - The plugin tool for Maven allowing Maven
 		          projects to process ANTLR grammars.

Each of these sub-directories also contains a file pom.xml that
controls the build of each sub-component (or module in Maven
parlance).

Build Parameters
-----------------

Alongside each pom.xml (other than for the antlr3-maven-plugin), you
will see that there is a file called antlr.config. This file is called
a filter and should contain a set of key/value pairs in the same
manner as Java properties files:

antlr.something="Some config thang!"

When the build of any component happens, any values in the
antlr.config for the master build file and any values in the
antlr.config file for each component are made available to the
build. This is mainly used by the resource processor, which will
filter any file it finds under: src/main/resources/** and replace any
references such as ${antlr.something} with the actual value at the
time of the build.

Building
--------

Building ANTLR is trivial, assuming that you have loaded Maven version
2.0.9 or better on to your build system and installed it as explained
here:

http://maven.apache.org/download.html

If you are unfamiliar with Maven (and even if you are), the best
resource for learning about it is The Definitive Guide:

http://www.sonatype.com/books/maven-book/reference/public-book.html

The instructions here assume that Maven is installed and working correctly.

If this is the first time you have built the ANTLR toolset, you will
possibly need to install the master pom in your local repository
(however the build may be able to locate this in the ANTLR snapshot or
release repository). If you try to build sub-modules on their own (as
in run the mvn command in the sub directory for that tool, such as
runtime/Java), and you receive a message that maven cannot find the
master pom, then execute this in the main (or release) directory:

mvn -N install

This command will install the master build pom in your local maven
repository (it's ~/.m2 on UNIX) and individual builds of sub-modules
will now work correctly.

To build then, simply cd into the master build directory
(e.g. $P4ROOT//code/antlr/main) and type:

mvn -Dmaven.test.skip=true

Assuming that everything is correctly installed and synchronized, then
ANTLR will build and skip any unit tests in the modules (the ANTLR
tool tests can take a long time).

This command will build each of the tools in the correct order and
will create the jar artifacts of all the components in your local
development Maven repository (which takes precedence over remote
repositories by default). At the end of the build you should see:

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] ANTLR Master build control POM ........................ SUCCESS [1.373s]
[INFO] Antlr 3 Runtime ....................................... SUCCESS [0.879s]
[INFO] ANTLR Grammar Tool .................................... SUCCESS [5.431s]
[INFO] Maven plugin for ANTLR V3 ............................. SUCCESS [1.277s]
[INFO] ANTLR gUnit ........................................... SUCCESS [1.566s]
[INFO] Maven plugin for gUnit ANTLR V3 ....................... SUCCESS [0.079s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11 seconds

However, unless you are using Maven exclusively in your projects, you
will most likely want to build the ANTLR Uber Jar, which is an
executable jar containing all the components that ANTLR needs to build
and run parsers (note that at runtime, you need only the runtime
components you use, such as the Java runtime and say stringtemplate).

Because the Uber jar is not something we want to deploy to Maven
repositories it is built with a special invocation of Maven:

mvn -Dmaven.test.skip=true package assembly:assembly

Note that Maven will appear to build everything twice, which is a
quirk of how it calculates the dependencies and makes sure it has
everything packaged up so it can build the uber-jar assembly.

Somewhere in the build output (towards the end), you will find a line
like this:

[INFO] Building jar: /home/jimi/antlrsrc/code/antlr/main/target/antlr-master-3.1.3-SNAPSHOT-completejar.jar

This is the executable jar that you need and you can either copy it
somewhere or, like me, you can create this script (assuming UNIX)
somewhere in your PATH:

#! /bin/bash
java -jar ~/antlrsrc/code/antlr/main/target/antlr-master-3.1.3-SNAPSHOT-completejar.jar $*

Version Numbering
-------------------

The first and Golden rule is that any pom files stored under the main
branch of the toolset should never be modified to contain a release
version number. They should always contain a.b.c-SNAPSHOT
(e.g. 3.1.3-SNAPSHOT). Only release branches should have their pom
version numbers set to a release version. You can release as many
SNAPSHOTS as you like, but only one release version. However, release
versions may be updated with a patch level: 3.1.3-1, 3.1.3-2 and so
on.

Fortunately, Maven helps us with the version numbering in a number of
ways. Firstly, the pom.xml files for the various modules do not
specify a version of the artifacts themselves. They pick up their
version number from the master build pom.  However, there is a catch,
because they need to know what version of the parent pom they inherit
from and so they DO mention the version number. However, this does
prevent accidentally releasing different versions of sub-modules than
the master pom describes.

Fortunately once again, Maven has a neat way of helping us change the
version.  All you need do is check out all the pom.xml files from
perforce, then modify the <version>a.b.c-SNAPSHOT</version> in the
master pom. When the version number is correct in the master pom, you
make sure your working directory is the location of the master pom and
type:

mvn versions:update-child-modules

This command will then update the child pom.xml files to reflect the
version number defined in the master pom.xml.

There is unfortunately one last catch here though and that is that the
antlr3-maven-plugin and the gunit-maven-plugin are not able to use the
parent pom. The reason for this is subtle but makes sense as doing so
would create a circular dependency between the ANTLR tool (which uses
the plugin to build its own grammar files), and the plugins (which
uses the tool to build grammar files and gunit to test).

This catch-22 situation means that the pom.xml file in the
antlr3-maven-plugin directory and the one in the gunit-maven-plugin
directory MUST be updated manually (or we must write a script to do
this).

Finally, we need to remember that because the tool is dependent on the
antlr3-maven-plugin and the plugin is itself dependent on the
tool, that we must manually update the versions of each that they
reference. So, when we bump the version of the toolset to say
3.1.4-SNAPSHOT, we need to change the antlr3-maven-plugin pom.xml and
the gunit-maven-plugin pom.xml to reference that version of the antlr
tool. The tool itself is always built with the prior released version
of the plugin, so when we release we must change the main branch of
the plugin to use the newly released version of the plugin. This is
covered in the release checklist.

Deploying
----------

Deploying the tools at the current version is relatively easy, but to
deploy to the ANTLR repositories (snapshot or release) you must have
been granted access to the antlr.org server and supplied an ssh
key. Few people will have this access of course.

Assuming that you have ssh access to antlr.org, then you will need to
do the following before deployment will authorize and work correctly
(UNIX assumed here):

$ eval `ssh-agent`
Agent PID nnnnn
$ ssh-add
Enter passphrase for /home/you/.ssh/id_rsa:
Identity added....

Next, because we do not publish access information for antlr.org, you
will need to configure the repository server names locally. You do
this by creating (or adding to) the file:

~/.m2/settings.xml

Which should look like this:

<?xml version="1.0" encoding="UTF-8"?>
<!--
    User-specific configuration for maven. Includes things that should not
    be distributed with the pom.xml file, such as developer identity, along with
    local settings, like proxy information. The default location for the
    settings file is ~/.m2/settings.xml
-->
<settings xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
    <servers>
        <server>
            <id>antlr-snapshot</id>
            <username>mavensync</username>
            <passphrase>passphrase for your private key</passphrase>
            <privateKey>/home/youruserlogin/.ssh/id_rsa</privateKey>
        </server>
        <server>
            <id>antlr-repo</id>
            <username>mavensync</username>
            <passphrase>passphrase for your private key</passphrase>
            <privateKey>/home/youruserlogin/.ssh/id_rsa</privateKey>
        </server>
    </servers>
</settings>

When this configuration is in place, you will be able to deploy the components,
either individually or from the master directory:

mvn -Dmaven.test.skip=true deploy

You will then see lots of information about checking existing version
information and so on, and the components will be deployed.

Note that so long as the artifacts are versioned with a.b.c-SNAPSHOT
then deployment will always be to the development snapshot
directory. When the artifacts are versioned with a release version
then deployment will be to the antlr.org release repository, which
will then be mirrored around the world. It is important not to deploy
a release until you have built and tested it to your satisfaction.

Release Checklist
------------------

Here is the procedure to use to make a release of ANTLR. Note that we
should really use the mvn release:release command, but the perforce
plugin for Maven is not commercial quality and I want to rewrite it.

For this checklist, let's assume that the current development version
of ANTLR is 3.1.3-SNAPSHOT. This means that it will probably (but not
necessarily) become release version 3.1.3 and that the development
version will bump to 3.1.4-SNAPSHOT.

0) Run a build of the main branch and check that it is builds and
   passes as many tests as you want it to.

1) First make a branch from main into the target release
   directory. Then submit this to perforce. You could change versions
   numbers before submitting, but doing that in separate stages will
   keep things sane;

--- Use main development branch from here ---

2) Before we deploy the release, we want to update the versions of the
   development branch, so we don't deploy what is now the new release
   as an older snapshot (this is not super important, but procedure is
   good right?).

   Check out all the pom.xml files (and if you are using any
   antlr.config parameters that must change, then do that too).

3) Edit the master pom.xml in the main directory and change the version from
   3.1.3-SNAPSHOT to 3.1.4-SNAPSHOT.

4) Edit the pom.xml file for antlr3-maven-plugin under the main
   directory and change the version from 3.1.3-SNAPSHOT to
   3.1.4-SNAPSHOT. Do the same for the pom.xml in the
   gunit-maven-plugin directory.

   Update the pom.xml for the archetype manually too.

5) Now (from the main directory), run the command:

         mvn versions:update-child-modules

      You should see:

         [INFO] [versions:update-child-modules]
         [INFO] Module: gunit
         [INFO]   Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT
         [INFO] Module: runtime/Java
         [INFO]   Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT
         [INFO] Module: tool
         [INFO]   Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT

6) Run a build of the main branch:

         mvn -Dmaven.test.skip=true

       All should be good.

7) Submit the pom changes of the main branch to perforce.

8) Deploy the new snapshot as a placeholder for the next release. It
   will go to the snapshot repository of course:

	  mvn -N deploy
          mvn -Dmaven.test.skip=true deploy

9) You are now finished with the main development branch and should change
   working directories to the release branch you made earlier.

--- Use release branch from here ---

10) Check out all the pom.xml files in the release branch (and if you are
    using any antlr.config parameters that must change, then do that too).

11) Edit the master pom.xml in the release-3.1.3 directory and change
    the version from 3.1.3-SNAPSHOT to 3.1.3.

12) Edit the pom.xml file for antlr3-maven-plugin under the
    release-3.1.3 directory and change the version from 3.1.3-SNAPSHOT
    to 3.1.3. Also change the version of the tool that the this
    pom.xml references from 3.1.3-SNAPSHOT to 3.1.3 as we are now
    releasing the plugin of course and it needs to reference the
    version we are about to release. You will find this reference in
    the dependencies section of the antlr3-maven-plugin pom.xml. Also
    change the version references in the pom for gunit-maven-plugin.

13)  Now (from the release-3.1.3 directory), run the command:

           mvn versions:update-child-modules

        You should see:

	[INFO] [versions:update-child-modules]
	[INFO] Module: gunit
	[INFO]   Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT,
	       now org.antlr:antlr-master:3.1.3
	[INFO] Module: runtime/Java
	[INFO]   Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT,
	       now org.antlr:antlr-master:3.1.3
	[INFO] Module: tool
	[INFO]   Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT,
	       now org.antlr:antlr-master:3.1.3

14)  Run a build of the release-3.1.3 branch:

           mvn   # Note I am letting unit tests run here!

        All should be good, or as good as it gets ;-)

15)  Submit the pom changes of the release-3.1.3 branch to perforce.

16)  Deploy the new release (this is it guys, make sure you are happy):

	  mvn -N deploy
          mvn -Dmaven.test.skip=true deploy

        Note that we must skip the tests as Maven will not let you
        deploy releases that fail any junit tests.

17) The final step is that we must update the main branch pom.xml for
     the tool to reference the newly release version of the
     antlr3-maven-plugin. This is because each release of ANTLR is
     built with the prior release of ANTLR, and we have just released
     a new version. Edit the pom.xml for the tool (main/tool/pom.xml)
     under the main (that's the MAIN branch, not the release branch)
     and find the dependency reference to the antlr plugin. If you
     just released say 3.1.3, then the tool should now reference
     version 3.1.3 of the plugin. Having done this, you should
     probably rebuild the main branch and let it run the junit
     tests. Later, I will automate this dependency update as mvn can
     do this for us.

18)  Having deployed the release to maven, you will want to create the
     uber jar for the new release, to make it downloadable from the
     antlr.org website. This is a repeat of the earlier described step
     to build the uber jar:

       mvn -Dmaven.test.skip=true package assembly:assembly

     MAven will produce the uber jar in the target directory:

	antlr-master-3.1.3-completejar.jar

     And this is the complete jar that can be downloaded from the web site. You
     may wish to produce an md5 checksum to go with the jar:

     md5sum target/antlr-master-3.1.3-completejar.jar
     xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx  target/antlr-master-3.1.4-SNAPSHOT-completejar.jar

     The command you just ran will also produce a second jar:

        antlr-master-3.1.3-src.jar

     This is the source code for everythign you just deployed and can
     be unjarred and built from scratch using the very procedures
     described here, which means you will now be reading this
     BUILD.txt file for ever.

19)  Reward anyone around you with good beer.


Miscellany
-----------

It was a little tricky to get all the interdependencies correct
because ANTLR builds itself using itself and the maven plugin
references the ANTLR Tool as well. Hence the maven tool is not a child
project of the master pom.xml file, even though it is built by it.

An observant person will not that when the assembly:assembly phase is
run, that it invokes the build of the ANTLR tool using the version of
the Maven plugin that it has just built, and this results in the
plugin using the version of ANTLR tool that it has just built. This is
safe because everything will already be up to date and so we package
up the version of the tool that we expect, but the Maven plugin we
deploy will use the correct version of ANTLR, even though there is
technically a circular dependency.

The master pom.xml does give us a way to cause the build of the ANTLR
tool to use itself to build itself. This is because in
dependencyManagement in the master pom.xml, we can reference the
current version of the Tool and the Maven plugin, even though in the
pom.xml for the tool itself refers to the previous version of the
plugin.

What happens is that if we first cd into the tool and maven
directories and build ANTLR, it will build itself with the prior
version and this will deploy locally (.m2). We can then clean build
from the master pom and when ANTLR asks for the prior version of the
tool, the master pom.xml will override it and build with the interim
versions we just built manually.

However, strictly speaking, we need a third build where we rebuild the
tool again with the version of the tool that was built with itself and
not deploy the version that was built by the version of itself that
was built by a prior version of itself. I decided that this was not
particularly useful and complicates things too much. Building with a
prior version of the tool is fine and if there was ever a need to, we
could release twice in quick succession.

I have occasionally seen the MAven reactor screw up (or perhaps it is
the ANTLR tool) when building. If this happens you will see an ANTLR
Panic - cannot find en.stg message. If this happens to you, then just
rerun the build and it will eventually work.

Jim Idle - March 2009