'Re: Maven surefire plugin: parallel configuration not running tests in parallel'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       maven-user
Subject:    Re: Maven surefire plugin: parallel configuration not running tests in parallel
From:       Tibor Digana <tibordigana () apache ! org>
Date:       2021-02-19 9:47:59
Message-ID: CAKuVzBRvBJQB=P9cvtr-=u8aNoCkJ9pgYnB=vuFn1dw86-tS5g () mail ! gmail ! com
[Download RAW message or body]


We will do our best as well. On one hand TeamCity uses flowId based on
Thread ID, the surefire plugin can use a similar mechanism we used before
in surefire-junit47-provider and we can identify the lines of system output
by Thread ID.

T

On Wed, Feb 17, 2021 at 6:31 PM Jay Crosley <jcrosley@axon.com> wrote:

> My plan is to run these serially for local development and in parallel for
> TeamCity, and for TeamCity I think it will pick out the logs using flowId,
> so hopefully that will work.
> 
> From: Tibor Digana <tibordigana@apache.org>
> Date: Wednesday, February 17, 2021 at 2:19 AM
> To: Maven Users List <users@maven.apache.org>
> Subject: Re: Maven surefire plugin: parallel configuration not running
> tests in parallel
> In case of combining JUnit5 and Surefire/Failsafe, the configuration
> parameters e.g. "parallel" and "threadCountClasses" are not bound to the
> native JUnit parameters "junit.jupiter.execution.parallel.enabled" because
> it was an agreement between Maven/JUnit teams and the solution became "by
> design". We can talk about a new concept and bind these parameters of
> course.
> 
> Regarding the parallel execution has it's own logging problems in plugin on
> the top of JUnit5 engine but that's another issue you may be facing. We are
> approaching the fix step by step.
> 
> Cheers
> Tibor
> 
> On Tue, Feb 16, 2021 at 11:04 PM Jay Crosley <jcrosley@axon.com> wrote:
> 
> > I'm trying to get junit5 tests to run in parallel using the maven
> surefire
> > plugin, as described on
> > 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__maven.apache.org_surefire_maven \
> -2Dsurefire-2Dplugin_examples_fork-2Doptions-2Dand-2Dparallel-2Dexecution.html&d=DwI \
> FaQ&c=0hefKdg9jtsMu47wpF0ovg&r=qyxGgg8iek4zUTwPHqQd2x5bP20ZI2bxMqb2S9cASmw&m=M_2Gjt2 \
> FtuqAHQQ0HhrtLwLB6txc8A381wmGzx0q2jw&s=9y-T5guZoywkTqejMKcpEcM60c0Eg_HNafGvBD39cxM&e=
>                 
> .
> > Despite configuration that looks correct, I can't get them to run in
> > parallel. I'll paste my configuration and what I've tried and experienced
> > so far. Any help is greatly appreciated!
> > 
> > My surefire plugin configuration looks like this:
> > 
> > 
> > <plugin>
> > <groupId>org.apache.maven.plugins</groupId>
> > <artifactId>maven-surefire-plugin</artifactId>
> > <version>2.21.0</version>
> > <dependencies>
> > <dependency>
> > <groupId>org.junit.platform</groupId>
> > <artifactId>junit-platform-surefire-provider</artifactId>
> > <version>1.2.0</version>
> > </dependency>
> > <dependency>
> > <groupId>org.junit.jupiter</groupId>
> > <artifactId>junit-jupiter-engine</artifactId>
> > <version>5.2.0</version>
> > </dependency>
> > </dependencies>
> > </plugin>
> > 
> > And I have a maven profile setup with additional configuration for our
> > integration tests, which includes the parallel configuration. The
> commented
> > out configurations indicate all the things I've tried.
> > 
> > 
> > <profile>
> > <id>integration-tests-local</id>
> > <build>
> > <plugins>
> > <plugin>
> > <groupId>org.apache.maven.plugins</groupId>
> > <artifactId>maven-surefire-plugin</artifactId>
> > <version>${surefire.version}</version>
> > <configuration>
> > <parallel>classes</parallel>
> > <threadCount>4</threadCount>
> > <useUnlimitedThreads>true</useUnlimitedThreads>
> > <!--
> > <useUnlimitedThreads>true</useUnlimitedThreads>-->
> > <!--              <threadCountClasses>4</threadCountClasses>-->
> > <!--              <threadCountMethods>4</threadCountMethods>-->
> > <!--              <threadCountSuites>4</threadCountSuites>-->
> > <!--              <threadCount>4</threadCount>-->
> > <parallelOptimized>false</parallelOptimized>
> > <!--
> > <parallelMavenExecution>true</parallelMavenExecution>-->
> > <perCoreThreadCount>false</perCoreThreadCount>
> > <excludes>
> > <exclude>none</exclude>
> > </excludes>
> > <includes>
> > <include>**/*IntegrationTests*.java</include>
> > </includes>
> > </configuration>
> > </plugin>
> > </plugins>
> > </build>
> > 
> > The project is structured with a parent pom.xml and several sub-projects.
> > The tests are in the "integration-tests" module, which pulls in the
> > surefire plugin with no additional configuration:
> > 
> > 
> > <plugin>
> > <artifactId>maven-surefire-plugin</artifactId>
> > </plugin>
> > 
> > I mention the sub-project because it means that the maven command I'm
> > running looks like this (I've tried with/without the "-T"):
> > 
> > 
> > mvn -T 4 test -e -pl integration-tests -am -Pintegration-tests-local
> > 
> > For the purposes of debugging, I've created 10 .java files named
> > TestIntegrationTests1.java through TestIntegrationTests10.java, each of
> > which has 10 unit tests all of which look like this:
> > 
> > 
> > package com.axon.scorpius.integration_tests;
> > 
> > import static org.junit.jupiter.api.Assertions.assertTrue;
> > 
> > import org.junit.jupiter.api.Test;
> > 
> > /**
> > * Tests.
> > */
> > public class TestIntegrationTests1 {
> > 
> > @Test
> > void test1() {
> > try {
> > Thread.sleep(1000);
> > } catch (InterruptedException ex) {
> > System.out.println("Interrupted exception: " + ex);
> > }
> > 
> > assertTrue(true);
> > }
> > 
> > 
> > 
> > … 9 identical tests
> > 
> > My hope is that when I run "mvn test" (I'm running locally in iTerm on a
> > Macbook), these 10 test classes will run in parallel (at least
> partially),
> > but they run serially, as seen here:
> > 
> > INFO] -------------------------------------------------------
> > [INFO]  T E S T S
> > [INFO] -------------------------------------------------------
> > [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests4
> > [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 11.057 s - in com.axon.scorpius.integration_tests.TestIntegrationTests4
> > [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests6
> > [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 11.029 s - in com.axon.scorpius.integration_tests.TestIntegrationTests6
> > [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests10
> > [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 11.025 s - in com.axon.scorpius.integration_tests.TestIntegrationTests10
> > [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests2
> > [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 11.027 s - in com.axon.scorpius.integration_tests.TestIntegrationTests2
> > [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests7
> > [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 11.032 s - in com.axon.scorpius.integration_tests.TestIntegrationTests7
> > [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests5
> > [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 11.027 s - in com.axon.scorpius.integration_tests.TestIntegrationTests5
> > [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests1
> > [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 11.032 s - in com.axon.scorpius.integration_tests.TestIntegrationTests1
> > [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests3
> > [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 11.036 s - in com.axon.scorpius.integration_tests.TestIntegrationTests3
> > [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests9
> > [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 11.013 s - in com.axon.scorpius.integration_tests.TestIntegrationTests9
> > [INFO] Running com.axon.scorpius.integration_tests.TestIntegrationTests8
> > [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 11.026 s - in com.axon.scorpius.integration_tests.TestIntegrationTests8
> > [INFO]
> > [INFO] Results:
> > [INFO]
> > [WARNING] Tests run: 113, Failures: 0, Errors: 0, Skipped: 2
> > 
> 



[prev in list] [next in list] [prev in thread] [next in thread]
Configure | About | News | Add a list | Sponsored by KoreLogic