| [The "BSD license"] |
| Copyright (c) 2010 Terence Parr |
| Maven Plugin - Copyright (c) 2009 Jim Idle |
| |
| All rights reserved. |
| |
| Redistribution and use in source and binary forms, with or without |
| modification, are permitted provided that the following conditions |
| are met: |
| 1. Redistributions of source code must retain the above copyright |
| notice, this list of conditions and the following disclaimer. |
| 2. Redistributions in binary form must reproduce the above copyright |
| notice, this list of conditions and the following disclaimer in the |
| documentation and/or other materials provided with the distribution. |
| 3. The name of the author may not be used to endorse or promote products |
| derived from this software without specific prior written permission. |
| |
| THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR |
| IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES |
| OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. |
| IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, |
| INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT |
| NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, |
| DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY |
| THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT |
| (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF |
| THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
| |
| ============================================================================ |
| |
| This file contains the build instructions for the ANTLR toolset as |
| of version 3.1.3 and beyond. |
| |
| The ANTLR toolset must be built using the Maven build system as |
| this build system updates the version numbers and controls the |
| whole build process. However, if you just want the latest build |
| and do not care to learn anything about Maven, then visit the 'target' |
| directories (for jars) under the depot mirror root here: |
| |
| http://antlr.org/depot |
| |
| If you are looking for the latest released version of ANTLR, then |
| visit the downloads page on the main antlr.org website. |
| |
| These instructions are mainly for the ANTLR development team, |
| though you are free to build ANTLR yourself of course. |
| |
| Source code Structure |
| ----------------------- |
| |
| The main development branch of ANTLR is stored within the Perforce SCM at: |
| |
| //depot/code/antlr/main/... |
| |
| release branches are stored in Perforce like so: |
| |
| //depot/code/antlr/release-3.1.3/... |
| |
| In this top level directory, you will find a master build file for |
| Maven called pom.xml and you will also note that there are a number of |
| subdirectories: |
| |
| tool - The ANTLR tool itself |
| runtime/Java - The ANTLR Java runtime |
| runtime/X - The runtime for language target X |
| gunit - The grammar test tool |
| antlr3-maven-plugin - The plugin tool for Maven allowing Maven |
| projects to process ANTLR grammars. |
| |
| Each of these sub-directories also contains a file pom.xml that |
| controls the build of each sub-component (or module in Maven |
| parlance). |
| |
| Build Parameters |
| ----------------- |
| |
| Alongside each pom.xml (other than for the antlr3-maven-plugin), you |
| will see that there is a file called antlr.config. This file is called |
| a filter and should contain a set of key/value pairs in the same |
| manner as Java properties files: |
| |
| antlr.something="Some config thang!" |
| |
| When the build of any component happens, any values in the |
| antlr.config for the master build file and any values in the |
| antlr.config file for each component are made available to the |
| build. This is mainly used by the resource processor, which will |
| filter any file it finds under: src/main/resources/** and replace any |
| references such as ${antlr.something} with the actual value at the |
| time of the build. |
| |
| Building |
| -------- |
| |
| Building ANTLR is trivial, assuming that you have loaded Maven version |
| 3.0.3 or better on to your build system and installed it as explained |
| here: |
| |
| http://maven.apache.org/download.html |
| |
| Note that the ANTLR toolset will ONLY build with version 3.0.3 of Maven |
| as of release 3.4. |
| |
| If you are unfamiliar with Maven (and even if you are), the best |
| resource for learning about it is The Definitive Guide: |
| |
| http://www.sonatype.com/books/maven-book/reference/public-book.html |
| |
| The instructions here assume that Maven is installed and working correctly. |
| |
| If this is the first time you have built the ANTLR toolset, you will |
| possibly need to install the master pom in your local repository |
| (however the build may be able to locate this in the ANTLR snapshot or |
| release repository). If you try to build sub-modules on their own (as |
| in run the mvn command in the sub directory for that tool, such as |
| runtime/Java), and you receive a message that maven cannot find the |
| master pom, then execute this in the main (or release) directory: |
| |
| mvn -N install |
| |
| This command will install the master build pom in your local maven |
| repository (it's ~/.m2 on UNIX) and individual builds of sub-modules |
| will now work correctly. |
| |
| To build then, simply cd into the master build directory |
| (e.g. $P4ROOT//code/antlr/main) and type: |
| |
| mvn -Dmaven.test.skip=true |
| |
| Assuming that everything is correctly installed and synchronized, then |
| ANTLR will build and skip any unit tests in the modules (the ANTLR |
| tool tests can take a long time). |
| |
| This command will build each of the tools in the correct order and |
| will create the jar artifacts of all the components in your local |
| development Maven repository (which takes precedence over remote |
| repositories by default). At the end of the build you should see: |
| |
| [INFO] ------------------------------------------------------------------------ |
| [INFO] Reactor Summary: |
| [INFO] ------------------------------------------------------------------------ |
| [INFO] ANTLR Master build control POM ........................ SUCCESS [1.373s] |
| [INFO] Antlr 3 Runtime ....................................... SUCCESS [0.879s] |
| [INFO] ANTLR Grammar Tool .................................... SUCCESS [5.431s] |
| [INFO] Maven plugin for ANTLR V3 ............................. SUCCESS [1.277s] |
| [INFO] ANTLR gUnit ........................................... SUCCESS [1.566s] |
| [INFO] Maven plugin for gUnit ANTLR V3 ....................... SUCCESS [0.079s] |
| [INFO] ------------------------------------------------------------------------ |
| [INFO] ------------------------------------------------------------------------ |
| [INFO] BUILD SUCCESSFUL |
| [INFO] ------------------------------------------------------------------------ |
| [INFO] Total time: 11 seconds |
| |
| However, unless you are using Maven exclusively in your projects, you |
| will most likely want to build the ANTLR Uber Jar, which is an |
| executable jar containing all the components that ANTLR needs to build |
| and run parsers (note that at runtime, you need only the runtime |
| components you use, such as the Java runtime and say stringtemplate). |
| |
| Because the Uber jar is not something we want to deploy to Maven |
| repositories it is built with a special invocation of Maven: |
| |
| mvn -Dmaven.test.skip=true package assembly:assembly |
| |
| Note that Maven will appear to build everything twice, which is a |
| quirk of how it calculates the dependencies and makes sure it has |
| everything packaged up so it can build the uber-jar assembly. |
| |
| Somewhere in the build output (towards the end), you will find a line |
| like this: |
| |
| [INFO] Building jar: /home/jimi/antlrsrc/code/antlr/main/target/antlr-master-3.4-SNAPSHOT-completejar.jar |
| |
| This is the executable jar that you need and you can either copy it |
| somewhere or, like me, you can create this script (assuming UNIX) |
| somewhere in your PATH: |
| |
| #! /bin/bash |
| java -jar ~/antlrsrc/code/antlr/main/target/antlr-master-3.4-SNAPSHOT-completejar.jar $* |
| |
| Version Numbering |
| ------------------- |
| |
| The first and Golden rule is that any pom files stored under the main |
| branch of the toolset should never be modified to contain a release |
| version number. They should always contain a.b.c-SNAPSHOT |
| (e.g. 3.1.3-SNAPSHOT). Only release branches should have their pom |
| version numbers set to a release version. You can release as many |
| SNAPSHOTS as you like, but only one release version. However, release |
| versions may be updated with a patch level: 3.1.3-1, 3.1.3-2 and so |
| on. |
| |
| Fortunately, Maven helps us with the version numbering in a number of |
| ways. Firstly, the pom.xml files for the various modules do not |
| specify a version of the artifacts themselves. They pick up their |
| version number from the master build pom. However, there is a catch, |
| because they need to know what version of the parent pom they inherit |
| from and so they DO mention the version number. However, this does |
| prevent accidentally releasing different versions of sub-modules than |
| the master pom describes. |
| |
| Fortunately once again, Maven has a neat way of helping us change the |
| version. All you need do is check out all the pom.xml files from |
| perforce, then modify the <version>a.b.c-SNAPSHOT</version> in the |
| master pom. When the version number is correct in the master pom, you |
| make sure your working directory is the location of the master pom and |
| type: |
| |
| mvn versions:update-child-modules |
| |
| This command will then update the child pom.xml files to reflect the |
| version number defined in the master pom.xml. |
| |
| There is unfortunately one last catch here though and that is that the |
| antlr3-maven-plugin and the gunit-maven-plugin are not able to use the |
| parent pom. The reason for this is subtle but makes sense as doing so |
| would create a circular dependency between the ANTLR tool (which uses |
| the plugin to build its own grammar files), and the plugins (which |
| uses the tool to build grammar files and gunit to test). |
| |
| This catch-22 situation means that the pom.xml file in the |
| antlr3-maven-plugin directory and the one in the gunit-maven-plugin |
| directory MUST be updated manually (or we must write a script to do |
| this). |
| |
| Finally, we need to remember that because the tool is dependent on the |
| antlr3-maven-plugin and the plugin is itself dependent on the |
| tool, that we must manually update the versions of each that they |
| reference. So, when we bump the version of the toolset to say |
| 3.1.4-SNAPSHOT, we need to change the antlr3-maven-plugin pom.xml and |
| the gunit-maven-plugin pom.xml to reference that version of the antlr |
| tool. The tool itself is always built with the prior released version |
| of the plugin, so when we release we must change the main branch of |
| the plugin to use the newly released version of the plugin. This is |
| covered in the release checklist. |
| |
| Deploying |
| ---------- |
| |
| Deploying the tools at the current version is relatively easy, but to |
| deploy to the ANTLR repositories (snapshot or release) you must have |
| been granted access to the Sonatype OSS repositories' ANTLR login. |
| Few people will have this access of course. |
| |
| Next, because we do not publish access information for antlr.org, you |
| will need to configure the repository server names locally. You do |
| this by creating (or adding to) the file: |
| |
| ~/.m2/settings.xml |
| |
| Which should look like this: |
| |
| <?xml version="1.0" encoding="UTF-8"?> |
| <settings> |
| <servers> |
| <server> |
| <id>sonatype-nexus-snapshots</id> |
| <username>xxxxxxx</username> |
| <password>xxxxxxx</password> |
| </server> |
| <server> |
| <id>sonatype-nexus-staging</id> |
| <username>xxxxxxx</username> |
| <password>xxxxxxx</password> |
| </server> |
| </servers> |
| </settings> |
| |
| When this configuration is in place, you will be able to deploy the components, |
| either individually or from the master directory: |
| |
| mvn -Dmaven.test.skip=true -Ddeplot deploy |
| |
| You will then see lots of information about checking existing version |
| information and so on, and the components will be deployed once you |
| supply the ANTLR public key passphrase to sign the jars. |
| |
| Note that so long as the artifacts are versioned with a.b.c-SNAPSHOT |
| then deployment will always be to the development snapshot |
| directory. When the artifacts are versioned with a release version |
| then deployment will be to the release stahinh repository, which |
| will then be mirrored around the world if closed and release. |
| The sonatype documentation should be consulted. |
| |
| Release Checklist |
| ------------------ |
| |
| Here is the procedure to use to make a release of ANTLR. Note that we |
| should really use the mvn release:release command, but the perforce |
| plugin for Maven is not commercial quality and I want to rewrite it. |
| |
| For this checklist, let's assume that the current development version |
| of ANTLR is 3.1.3-SNAPSHOT. This means that it will probably (but not |
| necessarily) become release version 3.1.3 and that the development |
| version will bump to 3.1.4-SNAPSHOT. |
| |
| 0) Run a build of the main branch and check that it is builds and |
| passes as many tests as you want it to. |
| |
| 1) First make a branch from main into the target release |
| directory. Then submit this to perforce. You could change versions |
| numbers before submitting, but doing that in separate stages will |
| keep things sane; |
| |
| --- Use main development branch from here --- |
| |
| 2) Before we deploy the release, we want to update the versions of the |
| development branch, so we don't deploy what is now the new release |
| as an older snapshot (this is not super important, but procedure is |
| good right?). |
| |
| Check out all the pom.xml files (and if you are using any |
| antlr.config parameters that must change, then do that too). |
| |
| 3) Edit the master pom.xml in the main directory and change the version from |
| 3.1.3-SNAPSHOT to 3.1.4-SNAPSHOT. |
| |
| 4) Edit the pom.xml file for antlr3-maven-plugin under the main |
| directory and change the version from 3.1.3-SNAPSHOT to |
| 3.1.4-SNAPSHOT. Do the same for the pom.xml in the |
| gunit-maven-plugin directory. |
| |
| Update the pom.xml for the archetype manually too. |
| |
| 5) Now (from the main directory), run the command: |
| |
| mvn versions:update-child-modules |
| |
| You should see: |
| |
| [INFO] [versions:update-child-modules] |
| [INFO] Module: gunit |
| [INFO] Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT |
| [INFO] Module: runtime/Java |
| [INFO] Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT |
| [INFO] Module: tool |
| [INFO] Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT |
| |
| 6) Run a build of the main branch: |
| |
| mvn -Dmaven.test.skip=true |
| |
| All should be good. |
| |
| 7) Submit the pom changes of the main branch to perforce. |
| |
| 8) Deploy the new snapshot as a placeholder for the next release. It |
| will go to the snapshot repository of course: |
| |
| mvn -N deploy |
| mvn -Dmaven.test.skip=true deploy |
| |
| 9) You are now finished with the main development branch and should change |
| working directories to the release branch you made earlier. |
| |
| --- Use release branch from here --- |
| |
| 10) Check out all the pom.xml files in the release branch (and if you are |
| using any antlr.config parameters that must change, then do that too). |
| |
| 11) Edit the master pom.xml in the release-3.1.3 directory and change |
| the version from 3.1.3-SNAPSHOT to 3.1.3. |
| |
| 12) Edit the pom.xml file for antlr3-maven-plugin under the |
| release-3.1.3 directory and change the version from 3.1.3-SNAPSHOT |
| to 3.1.3. Also change the version of the tool that the this |
| pom.xml references from 3.1.3-SNAPSHOT to 3.1.3 as we are now |
| releasing the plugin of course and it needs to reference the |
| version we are about to release. You will find this reference in |
| the dependencies section of the antlr3-maven-plugin pom.xml. Also |
| change the version references in the pom for gunit-maven-plugin. |
| |
| 13) Now (from the release-3.1.3 directory), run the command: |
| |
| mvn versions:update-child-modules |
| |
| You should see: |
| |
| [INFO] [versions:update-child-modules] |
| [INFO] Module: gunit |
| [INFO] Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT, |
| now org.antlr:antlr-master:3.1.3 |
| [INFO] Module: runtime/Java |
| [INFO] Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT, |
| now org.antlr:antlr-master:3.1.3 |
| [INFO] Module: tool |
| [INFO] Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT, |
| now org.antlr:antlr-master:3.1.3 |
| |
| 14) Run a build of the release-3.1.3 branch: |
| |
| mvn # Note I am letting unit tests run here! |
| |
| All should be good, or as good as it gets ;-) |
| |
| 15) Submit the pom changes of the release-3.1.3 branch to perforce. |
| |
| 16) Deploy the new release (this is it guys, make sure you are happy): |
| |
| mvn -N deploy |
| mvn -Dmaven.test.skip=true deploy |
| |
| Note that we must skip the tests as Maven will not let you |
| deploy releases that fail any junit tests. |
| |
| 17) The final step is that we must update the main branch pom.xml for |
| the tool to reference the newly release version of the |
| antlr3-maven-plugin. This is because each release of ANTLR is |
| built with the prior release of ANTLR, and we have just released |
| a new version. Edit the pom.xml for the tool (main/tool/pom.xml) |
| under the main (that's the MAIN branch, not the release branch) |
| and find the dependency reference to the antlr plugin. If you |
| just released say 3.1.3, then the tool should now reference |
| version 3.1.3 of the plugin. Having done this, you should |
| probably rebuild the main branch and let it run the junit |
| tests. Later, I will automate this dependency update as mvn can |
| do this for us. |
| |
| 18) Having deployed the release to maven, you will want to create the |
| uber jar for the new release, to make it downloadable from the |
| antlr.org website. This is a repeat of the earlier described step |
| to build the uber jar: |
| |
| mvn -Dmaven.test.skip=true package assembly:assembly |
| |
| MAven will produce the uber jar in the target directory: |
| |
| antlr-master-3.1.3-completejar.jar |
| |
| And this is the complete jar that can be downloaded from the web site. You |
| may wish to produce an md5 checksum to go with the jar: |
| |
| md5sum target/antlr-master-3.1.3-completejar.jar |
| xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx target/antlr-master-3.1.4-SNAPSHOT-completejar.jar |
| |
| The command you just ran will also produce a second jar: |
| |
| antlr-master-3.1.3-src.jar |
| |
| This is the source code for everythign you just deployed and can |
| be unjarred and built from scratch using the very procedures |
| described here, which means you will now be reading this |
| BUILD.txt file for ever. |
| |
| 19) Reward anyone around you with good beer. |
| |
| |
| Miscellany |
| ----------- |
| |
| It was a little tricky to get all the interdependencies correct |
| because ANTLR builds itself using itself and the maven plugin |
| references the ANTLR Tool as well. Hence the maven tool is not a child |
| project of the master pom.xml file, even though it is built by it. |
| |
| An observant person will not that when the assembly:assembly phase is |
| run, that it invokes the build of the ANTLR tool using the version of |
| the Maven plugin that it has just built, and this results in the |
| plugin using the version of ANTLR tool that it has just built. This is |
| safe because everything will already be up to date and so we package |
| up the version of the tool that we expect, but the Maven plugin we |
| deploy will use the correct version of ANTLR, even though there is |
| technically a circular dependency. |
| |
| The master pom.xml does give us a way to cause the build of the ANTLR |
| tool to use itself to build itself. This is because in |
| dependencyManagement in the master pom.xml, we can reference the |
| current version of the Tool and the Maven plugin, even though in the |
| pom.xml for the tool itself refers to the previous version of the |
| plugin. |
| |
| What happens is that if we first cd into the tool and maven |
| directories and build ANTLR, it will build itself with the prior |
| version and this will deploy locally (.m2). We can then clean build |
| from the master pom and when ANTLR asks for the prior version of the |
| tool, the master pom.xml will override it and build with the interim |
| versions we just built manually. |
| |
| However, strictly speaking, we need a third build where we rebuild the |
| tool again with the version of the tool that was built with itself and |
| not deploy the version that was built by the version of itself that |
| was built by a prior version of itself. I decided that this was not |
| particularly useful and complicates things too much. Building with a |
| prior version of the tool is fine and if there was ever a need to, we |
| could release twice in quick succession. |
| |
| I have occasionally seen the MAven reactor screw up (or perhaps it is |
| the ANTLR tool) when building. If this happens you will see an ANTLR |
| Panic - cannot find en.stg message. If this happens to you, then just |
| rerun the build and it will eventually work. |
| |
| Jim Idle - March 2009 |
| |