Prerequisites
Basic Concepts
For the below examples, a Maven <command> will typically be of the form mvn clean package install where mvn is the command being run, and clean/package/install/etc are phases of the Maven project you're building.
Maven includes several predefined phases to cover the default lifecycle:
- clean - clears out old build artifacts
- test - run the Maven surefire plugin to execute your tests and display the result (i.e. JUnit, TestNG, etc)
- package - download dependencies (if necessary) and build up new artifacts into the target/ folder
- install - installs built artifacts (assumes package) from your target/ folder into your local .m2 Maven repository, allowing local Maven projects to consume the built artifacts
- install-file - install a JAR file manually to your .m2 repo, allowing local Maven project to consume the JAR
- deploy - publish built artifacts (assumes package) to a remote Maven repository, allowing local Maven projects to consume the built artifacts
- deploy-file - publish a standalone JAR file manually to a remote Maven repository, allowing other remote Maven projects to consume the JAR
For a full list of these lifecycle goals: https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html#Lifecycle_Reference
How it Works
Every Maven project contains a file called pom.xml at the project root. This file is the POM or Project Object Model, which tells Maven everything it needs to know about the project's dependencies and build steps.
Running a Maven command will parse the current project's POM to determine available phases/goals, and execute them according to the details defined within.
The pom.xml can also link off to various other XML documents describing various pluggable aspects of the build process:
- build.xml: if you're using the Maven Antrun Plugin, this file tells Ant how to execute
- assembly.xml: If you're using the Maven Assembly Plugin, this file tells Maven how to package your application (JAR, WAR, ZIP, etc)
Nested Projects
POMs can also exist in subfolders of the root. Every subproject's POM inherits the values defined by the parent, creating a nested hierarchy.
Values in subprojects will override values in the parent project, so we must first compute an "effective POM" by walking down the POM tree from root project to our target subproject and collect all values encountered.
Building Artifacts (Basic)
Now that you've heard the basics, let's try it out!
If you're lucky, you'll only ever need to know a single command to use Maven...
Via Maven
The following command will run Maven's clean, package, and install phases natively:
$ mvn clean package install
This is a very common command to build a Maven project from source, and install its built artifacts into your .m2 cache for local consumption.
Via Docker
The following command will run the same phases within a Docker container, without needing to install Maven on your machine:
$ docker run -it --rm -v $(pwd):/build -w /build maven:3-jdk-8 mvn clean package install
Installing Artifacts (Intermediate)
Sometimes, your dependency might not be hosted publically or may be unavailable for download.
If this is the case, your build will almost certainly fail with "Unable to locate artifact" errors if you don't already have the artifacts in your local cache.
You can manually install such missing dependencies using the following command:
$ mvn install:install-file -Dfile=./indri-5.11.jar -DgroupId=edu.illinois.lis -DartifactId=indri -Dversion=5.11 -Dpackaging=jar
For example, the above command will install indri-5.11.jar into your local Maven cache, allowing your <dependencies> section to be able to resolve it.
Deploying Artifacts (Advanced)
Most people will have stopped reading by now, but if you're part of the unlucky few, then you have my condolences.
Self-Hosted via Docker
To run your own test Nexus repository via the official Docker image:
$ docker run -d -p 8081:8081 --name nexus sonatype/nexus:oss
You should then be able to follow the steps below (untested) using the IP/hostname of this machine and port 8081.
You can test it with the following command:
$ curl -u admin:admin123 http://localhost:8081/service/metrics/ping
Open-Source Software Repository Hosting (OSSRH)
See http://central.sonatype.org/pages/ossrh-guide.html
For a more long-term solution for hosting your (open-source) Maven artifacts, you can follow these steps to deploy them to OSSRH.
Choosing Project Coordinates
See http://central.sonatype.org/pages/choosing-your-coordinates.html
First and foremost, your Maven groupId should be the reverse of a FQDN over which you have some management rights. For example, edu.illinois.lis or org.nationaldataservice or org.ndslabs.* might be acceptable groupIds for the biocaddie code.
Ideally, your Maven groupId and Java packages should match or at the very least be fairly similar.
Create a Ticket on the Sonatype JIRA
If you don't already have one, create an account on the Sonatype JIRA
Plug your JIRA account credentials into the <servers> block of your settings.xml file (on OS X, this was located at /usr/local/Cellar/maven/3.5.0/libexec/conf/settings.xml):
<servers>
<server>
<id>ossrh</id>
<username>YOUR_JIRA_USERNAME</username>
<password>YOUR_JIRA_PASSWORD</password>
</server>
</servers>
Next, create a New Project ticket on the Sonatype JIRA describing the artifacts that you plan to deploy to OSSRH.
The Sonatype team will reach out to confirm that you "own" the domain used for the groupId (see above), and can answer any questions you might have.
The turnaround time was really quick for my ticket, and they will respond in the comments with next steps or problems (if any arise).
Deploying a Single Artifact
Once your project ticket has been approved, you can use a syntax similar to that of install-file to deploy a single artifact to a remote Nexus repo:
# Deploying a SNAPSHOT (non-release) artifact $ mvn deploy:deploy-file -Dfile=./indri-5.11.jar -DgroupId=edu.illinois.lis -DartifactId=indri -Dversion=5.11-SNAPSHOT -Dpackaging=jar -Durl=https://oss.sonatype.org/content/repositories/snapshots/ -DrepositoryId=ossrh # Deploying a release (non-SNAPSHOT) artifact $ mvn deploy:deploy-file -Dfile=./indri-5.11.jar -DgroupId=edu.illinois.lis -DartifactId=indri -Dversion=5.11 -Dpackaging=jar -Durl=https://oss.sonatype.org/service/local/staging/deploy/maven2/ -DrepositoryId=ossrh
NOTE: SNAPSHOTs will overwrite each other... need to determine how to deploy a set of SNAPSHOT artifacts. Maybe manually bundling them up before calling deploy-file?
Configuring your POM for Deployment
Once your project ticket has been approved, you can add the following block to your project's pom.xml file:
<distributionManagement>
<!-- When your version ends with "-SNAPSHOT", it will be deployed here -->
<snapshotRepository>
<id>ossrh</id>
<url>https://oss.sonatype.org/content/repositories/snapshots</url>
</snapshotRepository>
<!-- When your version does not end with "-SNAPSHOT", it will be deployed here -->
<repository>
<id>ossrh</id>
<url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
</repository>
</distributionManagement>
NOTE: The <id> here should match the <servers> entry in your settings.xml.
You should now be able to deploy to the above repositories by running the following command:
mvn deploy
Syncing to Maven Central
OSSRH will even sync your released artifacts with Maven Central on your behalf, but still has a strict list of requirements for artifacts synced to Maven Central.
See http://central.sonatype.org/pages/requirements.html
In general, the requirements are as follows:
- Final pom.xml must include:
- Project coordinates: groupId / artifactId / version
- Project name and description
- License information
- Developer information
- SCM information
- Final pom.xml must NOT include:
- <repositories> tags
- <pluginRepositories> tags
- Final bundle.jar must include:
- Project POM (i.e. ir-utils-0.1.0.pom)
- Compiled JARs (i.e. ir-utils-0.1.0.jar)
- JavaDoc JAR alongside each compiled JAR (i.e. ir-utils-0.1.0-javadoc.jar)
- Sources JAR alongside each compiled JAR (i.e. ir-utils-0.1.0-sources.jar)
- GPG signatures for all of the above (i.e. ir-utils-0.1.0.jar.asc, ir-utils-0.1.0-javadoc.jar.asc, etc)
You will then need to comment back on your OSSRH ticket after promoting your first release for them to enable sync to Maven Central.
After that, any future releases that you promote from the staging repository will be automatically synced to Maven Central (within a few hours).
Generating JavaDoc via Maven
Add the following to your pom.xml to generate a *-javadoc.jar alongside each JAR that is output during the package phase of your build:
<build>
<plugins>
<!-- More plugins... -->
<!-- Maven JavaDoc Plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.10.4</version>
<configuration>
<show>private</show>
<nohelp>true</nohelp>
</configuration>
<executions>
<execution>
<id>attach-javadocs</id>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- More plugins... -->
</plugins>
</build>
Generating Sources via Maven
Add the following to your pom.xml to generate a *-sources.jar alongside each JAR that is output during the package phase of your build:
<build>
<plugins>
<!-- More plugins... -->
<!-- Maven Sources Plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<version>3.0.1</version>
<executions>
<execution>
<id>attach-sources</id>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- More plugins... -->
</plugins>
</build>
Signing Your Artifacts via Maven
See http://central.sonatype.org/pages/working-with-pgp-signatures.html
This will ensure that consumers of your artifacts can verify the authenticity of those that they download (similar to MD5 checksum).
Add the following to your pom.xml to generate a *.asc alongside each output file during the verify phase of your build:
<build>
<plugins>
<!-- More plugins... -->
<!-- Maven GPG Plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-gpg-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<id>sign-artifacts</id>
<phase>verify</phase>
<goals>
<goal>sign</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- More plugins... -->
</plugins>
</build>
NOTE: You will now be asked for your passphrase when running "mvn deploy"
NOTE: If your build fails with "gpg: signing failed: Inappropriate ioctl for device", run the following before the retrying the mvn command:
export GPG_TTY=$(tty)
Generate a Keypair
If you don't already have one, you will need to generate a GPG key with which you can sign your artifacts.
You will need the gnupg command line tool (on OSX, this can be installed via brew):
$ gpg --gen-key
Upload to Keyserver
Once you have generated a keypair, you will need to upload your public key to a public keyserver.
To list existing keys:
$ gpg --list-keys
/Users/<USERNAME>/.gnupg/pubring.kbx
----------------------------------
pub rsa2048 2017-05-05 [SC] [expires: 2019-05-05]
<YOUR_KEY_ID_WHICH_WILL_BE_FAIRLY_LONG>
uid [ultimate] Craig Willis (https://github.com/craig-willis) <willis8@illinois.edu>
uid [ultimate] Garrick Sherman (https://github.com/gtsherman) <gsherma2@illinois.edu>
uid [ultimate] Mike Lambert (https://github.com/bodom0015) <lambert8@illinois.edu>
sub rsa2048 2017-05-05 [E] [expires: 2019-05-05]
Then execute the following command to upload your public key:
$ gpg --keyserver hkp://pool.sks-keyservers.net --send-keys <YOUR_KEY_ID_WHICH_WILL_BE_FAIRLY_LONG>
where <YOUR_KEY_ID_WHICH_WILL_BE_FAIRLY_LONG> is pulled from the output above
Manually Signing a Single File
To sign a file with your GPG key:
$ gpg -ab ir-utils-0.1.0.jar
NOTE: You will be asked to enter your GPG key's passphrase
This will output a .asc file that should be included alongside the file that produced the signature.
Verifying GPG Signatures
As a consumer, you can verify the authenticity of the Maven artifacts by using the gpg command line tool:
$ gpg --verify target/ir-utils-0.1.0.jar.asc gpg: assuming signed data in 'target/ir-utils-0.1.0.jar' gpg: Signature made Thu Jun 1 21:21:16 2017 CDT gpg: using RSA key <YOUR_KEY_ID_WHICH_WILL_BE_FAIRLY_LONG> gpg: Good signature from "Craig Willis (https://github.com/craig-willis) <willis8@illinois.edu>" [ultimate] gpg: aka "Garrick Sherman (https://github.com/gtsherman) <gsherma2@illinois.edu>" [ultimate] gpg: aka "Mike Lambert (https://github.com/bodom0015) <lambert8@illinois.edu>" [ultimate]
This will output the signature data from the .asc file, which should give some confidence that the artifacts were not modified or replaced by a malicious party.
Bringing It All Together
If you followed all of the above steps, then running the following command should build up all of the necessary artifacts and deploy them to the appropriate repository:
$ mvn clean package gpg:sign install:install deploy:deploy
This single command will:
- Clear out your existing build artifacts
- Compile and package up your code into a JAR
- Generate JavaDoc from your project and package it as a JAR
- Package up your source code as a JAR
- Sign the JAR and POM files with your GPG key
- Install the resulting artifacts into your local Maven cache
- Deploy the resulting artifacts into the remote Nexus repository
Staging a New Release
Bundle your build artifacts (JAR / POM / ASC files) into a single bundle.jar to upload them to OSSRH staging:
$ jar cvf bundle.jar *.pom *.jar *.asc
Then log into https://oss.sonatype.org/ with your JIRA username / password and choose "Staging Upload" on the left side.
Choose "Artifact Bundle" from the dropdown, and select your bundle.jar for upload.
After uploading, select "Staging Repositories" on the left side, and locate your bundle in the list.
To abort the process, you can choose "Drop" to delete your staged bundle.
Select the repository and choose "Close" at the top. This will run a set of checks to verify that the artifacts are of sufficient quality to upload to Maven Central.
Insert snippet about Maven Release Plugin, which supposedly helps to automate the above processes
<build>
<plugins>
<!-- More plugins... -->
<!-- Maven Release Plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-release-plugin</artifactId>
<version>2.5.3</version>
<configuration>
<mavenExecutorId>forked-path</mavenExecutorId>
<useReleaseProfile>false</useReleaseProfile>
<arguments>-Psonatype-oss-release</arguments>
</configuration>
</plugin>
<!-- More plugins... -->
</plugins>
</build>
Third-Party Dependencies
To sync third-party dependencies
- Ask for the developers' permission to publish their code, consult the license, etc
- DO NOT publish another developer's code without their permission!
- Locate the source code (if possible)
If found, generate a JAR containing the source code:
jar -cvf indri-5.11-sources.jar *
If found, generate JavaDoc HTML from the source code and bundle it into a separate JAR:
mkdir -p ./target/javadoc javadoc -d ./target/javadoc lemurproject.indri cd target/javadoc jar -cvf indri-5.11-javadoc.jar *
NOTE: Scala projects can supposedly skip this step
Locate compiled binaries (you can choose to download them, or build them yourself if you are able)
Author a simple POM for this dependency. Below is an example POM.xml that might be used for edu.illinois.lis.indri:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>edu.illinois.lis</groupId> <artifactId>indri</artifactId> <version>5.11</version> <name>Indri 5.11</name> <description>Indri is a text search engine developed at UMass. It is a part of the Lemur project.</description> <url>https://sourceforge.net/projects/lemur/</url> <licenses> <!-- Indri actually uses BSD-old / "original BSD", but its link is no longer available via opensource.org. This is likely due to the controversy surrounding the 4th clause --> <license> <name>BSD-3-Clause</name> <url>https://opensource.org/licenses/BSD-3-Clause</url> </license> </licenses> <scm> <connection>scm:svn:https://svn.code.sf.net/p/lemur/code/ lemur-code</connection> <developerConnection>scm:svn:https://svn.code.sf.net/p/lemur/code/ lemur-code</developConnection> <url>scm:svn:https://sourceforge.net/p/lemur/code/HEAD/tree/indri/tags/release-5.11/</url> </scm> <developers> <developer> <name>David Fisher</name> <email>dfisher@cs.umass.edu</email> <organization>University of Massachusetts at Amherst</organization> <organizationUrl>http://www.umass.edu/</organizationUrl> </developer> <developer> <name>Stephen Harding</name> <email>harding@hobart.cs.umass.edu</email> <organization>University of Massachusetts at Amherst</organization> <organizationUrl>http://www.umass.edu/</organizationUrl> </developer> <developer> <name>Cameron VandenBerg</name> <email>cmw2@andrew.cmu.edu</email> <organization>Carnegie Mellon University</organization> <organizationUrl>http://www.cmu.edu/</organizationUrl> </developer> </developers> </project>Sign all artifacts with your GPG key (*.pom, *.jar, *-sources.jar, *-javadoc.jar):
gpg -ab indri-5.11.pom gpg -ab indri-5.11.jar gpg -ab indri-5.11-sources.jar gpg -ab indri-5.11-javadoc.jar
This will produce a .asc file for each artifact
lambert8:5.11 lambert8$ ls -al total 65168 drwxr-xr-x 14 lambert8 staff 476 Jun 2 16:50 . drwxr-xr-x 5 lambert8 staff 170 Jun 2 15:48 .. -rw-r--r-- 1 lambert8 staff 93941 Jun 2 15:18 indri-5.11-javadoc.jar -rw-r--r-- 1 lambert8 staff 488 Jun 2 15:42 indri-5.11-javadoc.jar.asc -rw-r--r-- 1 lambert8 staff 16384085 Jun 2 15:41 indri-5.11-sources.jar -rw-r--r-- 1 lambert8 staff 488 Jun 2 15:42 indri-5.11-sources.jar.asc -rw-r--r-- 1 lambert8 staff 259518 May 22 10:54 indri-5.11.jar -rw-r--r-- 1 lambert8 staff 488 Jun 2 15:42 indri-5.11.jar.asc -rw-r--r-- 1 lambert8 staff 2035 Jun 2 16:49 indri-5.11.pom -rw-r--r-- 1 lambert8 staff 488 Jun 2 16:49 indri-5.11.pom.asc
Bundle everything up and Stage this release in the same way as described above:
jar -cvf indri-5.11-bundle.jar *.jar *.pom *.asc LICENSE