Maven Tips

Jesse Anderson
January 4, 2017
Blog, Business, Data Engineering
One Comment

Blog Summary: (AI Summaries by Summarizes)

Working with complex and multi-module Maven projects can be challenging.
To list all modules in a project, run the command `mvn help:all-profiles`.
The output will show the project's reactor build order and list all profiles for each module.
The command can be helpful for navigating and understanding the structure of a multi-module project.
The post lists various Profile Ids for different projects in Apache Beam.

Working with complex and multi-module Maven projects can be a handful.

These are a few tips to make that easier. I’m going to use Apache Beam as an example of a multi-module Maven project.

The first helpful command is to list all of the modules. To do this run mvn help:all-profiles and that will output:

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] Apache Beam :: Parent
[INFO] Apache Beam :: SDKs :: Java :: Build Tools
[INFO] Apache Beam :: SDKs
[INFO] Apache Beam :: SDKs :: Java
[INFO] Apache Beam :: SDKs :: Java :: Core
[INFO] Apache Beam :: Runners
[INFO] Apache Beam :: Runners :: Core Java
[INFO] Apache Beam :: Runners :: Direct Java
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow
[INFO] Apache Beam :: SDKs :: Java :: IO
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform
[INFO] Apache Beam :: SDKs :: Java :: IO :: HDFS
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples - Java 8
[INFO] Apache Beam :: SDKs :: Java :: Extensions
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests
[INFO] Apache Beam :: Runners :: Flink
[INFO] Apache Beam :: Runners :: Flink :: Core
[INFO] Apache Beam :: Runners :: Flink :: Examples
[INFO] Apache Beam :: Runners :: Spark
[INFO] Apache Beam :: Runners :: Apex
[INFO] Apache Beam :: Examples
[INFO] Apache Beam :: Examples :: Java
[INFO] Apache Beam :: Examples :: Java 8
[INFO] Listing Profiles for Project: org.apache.beam:beam-parent:pom:0.5.0-SNAPSHOT
  Profile Id: doclint-java8-disable (Active: true , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-build-tools:jar:0.5.0-SNAPSHOT
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-parent:pom:0.5.0-SNAPSHOT
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-parent:pom:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: true , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-core:jar:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-runners-parent:pom:0.5.0-SNAPSHOT
  Profile Id: release (Active: false , Source: pom)
  Profile Id: runnable-on-service-tests (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-runners-core-java:jar:0.5.0-SNAPSHOT
  Profile Id: release (Active: false , Source: pom)
  Profile Id: runnable-on-service-tests (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-runners-direct-java:jar:0.5.0-SNAPSHOT
  Profile Id: release (Active: false , Source: pom)
  Profile Id: runnable-on-service-tests (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-runners-google-cloud-dataflow-java:jar:0.5.0-SNAPSHOT
  Profile Id: release (Active: false , Source: pom)
  Profile Id: runnable-on-service-tests (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-io-parent:pom:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-io-hdfs:jar:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-io-jms:jar:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-io-kafka:jar:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-io-kinesis:jar:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-io-mongodb:jar:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-io-jdbc:jar:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-io-mqtt:jar:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-maven-archetypes-parent:pom:0.5.0-SNAPSHOT
  Profile Id: beam-sdks-java-archetypes-examples (Active: true , Source: pom)
  Profile Id: beam-sdks-java-archetypes-examples-java8 (Active: true , Source: pom)
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-maven-archetypes-starter:maven-archetype:0.5.0-SNAPSHOT
  Profile Id: beam-sdks-java-archetypes-examples (Active: false , Source: pom)
  Profile Id: beam-sdks-java-archetypes-examples-java8 (Active: false , Source: pom)
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-maven-archetypes-examples:maven-archetype:0.5.0-SNAPSHOT
  Profile Id: beam-sdks-java-archetypes-examples (Active: false , Source: pom)
  Profile Id: beam-sdks-java-archetypes-examples-java8 (Active: false , Source: pom)
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-maven-archetypes-examples-java8:maven-archetype:0.5.0-SNAPSHOT
  Profile Id: beam-sdks-java-archetypes-examples (Active: false , Source: pom)
  Profile Id: beam-sdks-java-archetypes-examples-java8 (Active: false , Source: pom)
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-extensions-parent:pom:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-extensions-join-library:jar:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-extensions-sorter:jar:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-sdks-java-java8tests:jar:0.5.0-SNAPSHOT
  Profile Id: java8-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-runners-flink-parent:pom:0.5.0-SNAPSHOT
  Profile Id: release (Active: false , Source: pom)
  Profile Id: runnable-on-service-tests (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-runners-flink_2.10:jar:0.5.0-SNAPSHOT
  Profile Id: local-runnable-on-service-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: runnable-on-service-tests (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-runners-flink_2.10-examples:jar:0.5.0-SNAPSHOT
  Profile Id: disable-runnable-on-service-tests (Active: true , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: runnable-on-service-tests (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-runners-spark:jar:0.5.0-SNAPSHOT
  Profile Id: jacoco (Active: false , Source: pom)
  Profile Id: local-runnable-on-service-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: runnable-on-service-tests (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-runners-apex:jar:0.5.0-SNAPSHOT
  Profile Id: local-runnable-on-service-tests (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: runnable-on-service-tests (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-examples-parent:pom:0.5.0-SNAPSHOT
  Profile Id: java8 (Active: true , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-examples-java:jar:0.5.0-SNAPSHOT
  Profile Id: direct-runner (Active: true , Source: pom)
  Profile Id: apex-runner (Active: false , Source: pom)
  Profile Id: flink-runner (Active: false , Source: pom)
  Profile Id: spark-runner (Active: false , Source: pom)
  Profile Id: dataflow-runner (Active: false , Source: pom)
  Profile Id: jenkins-precommit (Active: false , Source: pom)
  Profile Id: disable-streaming-ITs (Active: false , Source: pom)
  Profile Id: java8 (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)
Listing Profiles for Project: org.apache.beam:beam-examples-java8:jar:0.5.0-SNAPSHOT
  Profile Id: include-runners (Active: true , Source: pom)
  Profile Id: java8 (Active: false , Source: pom)
  Profile Id: release (Active: false , Source: pom)
  Profile Id: doclint-java8-disable (Active: false , Source: pom)
  Profile Id: java7-packageinfo (Active: false , Source: pom)
  Profile Id: eclipse-jdt (Active: false , Source: pom)
  Profile Id: apache-release (Active: false , Source: pom)

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Beam :: Parent .............................. SUCCESS [  0.234 s]
[INFO] Apache Beam :: SDKs :: Java :: Build Tools ......... SKIPPED
[INFO] Apache Beam :: SDKs ................................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java ........................ SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Core ................ SKIPPED
[INFO] Apache Beam :: Runners ............................. SKIPPED
[INFO] Apache Beam :: Runners :: Core Java ................ SKIPPED
[INFO] Apache Beam :: Runners :: Direct Java .............. SKIPPED
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO .................. SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: HDFS .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS ........... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka ......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB ....... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: JDBC .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: IO :: MQTT .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes .... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples - Java 8 SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions .......... SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Sorter  SKIPPED
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests ........ SKIPPED
[INFO] Apache Beam :: Runners :: Flink .................... SKIPPED
[INFO] Apache Beam :: Runners :: Flink :: Core ............ SKIPPED
[INFO] Apache Beam :: Runners :: Flink :: Examples ........ SKIPPED
[INFO] Apache Beam :: Runners :: Spark .................... SKIPPED
[INFO] Apache Beam :: Runners :: Apex ..................... SKIPPED
[INFO] Apache Beam :: Examples ............................ SKIPPED
[INFO] Apache Beam :: Examples :: Java .................... SKIPPED
[INFO] Apache Beam :: Examples :: Java 8 .................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.480 s
[INFO] Finished at: 2016-12-28T20:47:11-05:00
[INFO] Final Memory: 26M/162M
[INFO] ------------------------------------------------------------------------

Choosing a Module

From there, you can choose which module you want to start working with. In my case, I wanted to work with beam-sdks-java-core. From here, we can run a Maven command to start working with that module mvn verify -pl org.apache.beam:beam-sdks-java-core.

Unit Testing

For some of the unit tests in Beam, there are categories. These categories need a runner. In Beam, these are marked with @Category(NeedsRunner.class).

When running unit tests in IntelliJ, you’ll need to make a configuration change. Go to Run->Edit Configurations. Change Use classpath of module to beam-runners-direct-java. This will make the unit test run with the direct runner for the test. The image below shows what it should look like:

screen-shot-2016-12-29-at-9-59-32-am

To run a specific unit test from the command line run:

mvn -Dtest=org.apache.beam.sdk.transforms.RegexTest \
-DfailIfNoTests=false \
-pl runners/direct-java -am integration-test

Using an IDE is by far the easiest way to run individual unit tests.

(Update for improved searching for the error) If you’re getting an error that looks like this:

java.lang.IllegalStateException: The test was annotated with a [@NeedsRunner] / [@ValidatesRunner]
while the runner was set to [CrashingRunner]. Please re-check 
your configuration.

The fix is to add the -pl or change the classpath as shown above.

Thanks to Dan Halperin for his help.

Maven Tips

Choosing a Module

Unit Testing

Related Posts

Unapologetically Technical Episode 10 – Michael Drogalis

Why Most Data Projects Fail & How to Avoid It at GOTO 2023

Unapologetically Technical Episode 9 – Gunnar Morling

Unapologetically Technical Episode 8 – Tom Scott

The State of Data Engineering at Data Day Texas 2024

Unapologetically Technical Episode 7 – Stephane Derosiaux

The Difference Between Learning and Doing

Unapologetically Technical Episode 6 – Matteo Merli

The Data Discovery Team

Join the Newsletter