[Mvn test on the basis of testing and opening] 1. Use maven-surefire-plugin to complete the basic execution management and stepping on test cases

[Mvn test on the basis of testing and opening] 1. Use maven-surefire-plugin to complete the basic execution management and stepping on test cases

1. Demand

  • 1. When executing automated test cases, I only want to specify a certain test class, or a certain method, or a certain type of use case, etc. What should I do?
  • 2. I want to perform continuous integration with Jenkins, but the use case cannot be executed in the IDE, what should I do?

At this time, we need maven to debut. Using maven maven-surefire-pluginplug-ins can help us accomplish the above goals! It can manage the use cases we want to execute in the form of command lines.

2. Environment

2.1 Environmental requirements

  • Maven 3.x +
  • JDK 1.7 +

2.2 Support framework

For our testing, several use case management frameworks that we often contact are supported (here refers to the Java technology stack)

  • Using TestNG
  • Using JUnit 5 Platform
  • Using JUnit

2.3 Related dependencies

This article will take Junit5 as an example for research and learning, so the following dependency additions are also subject to Junit5

Commonly used Junit5 related dependencies:

<dependency>
   <groupId>org.junit.platform</groupId>
    <artifactId>junit-platform-launcher</artifactId>
    <version>1.6.2</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.junit.platform</groupId>
    <artifactId>junit-platform-runner</artifactId>
    <version>1.5.1</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.6.2</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.junit.vintage</groupId>
    <artifactId>junit-vintage-engine</artifactId>
    <version>5.6.2</version>
    <scope>test</scope>
</dependency>
 

maven-surefire-plugin:

<build>
   <plugins>
       <plugin>
           <groupId>org.apache.maven.plugins</groupId>
           <artifactId>maven-surefire-plugin</artifactId>
           <version>2.22.0</version>
       </plugin>
       <plugin>
           <groupId>org.apache.maven.plugins</groupId>
           <artifactId>maven-compiler-plugin</artifactId>
           <version>3.8.1</version>
           <configuration>
               <source>8</source>
               <target>8</target>
           </configuration>
            </plugin>
   </plugins>
</build>
 

Specify the compiled jdk version:

<properties>
    <aspectj.version>1.8.10</aspectj.version>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
</properties>
 

3. Specify the test class

Existing test classes with the following structure, on this basis for demonstration

3.1 mvn test

Use the mvn testcommand to execute all the use cases at once, and you can see the results in the console after the execution:

After the execution, we open it target, and all maventhe results of the execution will be under the targetdirectory surefire-reports:
among them:

  • xml : can be used for Jenkinsparsing, the file corresponds to the test class, and the content inside represents the execution of the test case.
  • txt : command line output of test results

3.2 mvn -Dtest=${TestClass} test

Use the mvn -Dtest=${TestClass}command to specify the running test class, for example

MavenDemo1TestThe use case I want to run here can be used mvn -Dtest=MavenDemo1Test test:

import org.junit.jupiter.api.Test;

public class MavenDemo1Test {

    @Test
    void test1(){
        System.out.println("MavenDemo1Test-test1");
    }

    @Test
    void test2(){
        System.out.println("MavenDemo1Test-test2");
    }
}
 

Perform the test:

$ mvn -Dtest=MavenDemo1Test test
 

Test Results:

3.3 mvn -Dtest=${TestClass1},${TestClass2} test

We can also specify multiple test classes to be executed separately, for example, here I want to execute MavenDemo1Testand MavenDemo2Test:

mvn -Dtest=MavenDemo1Test,MavenDemo2Test test
 

Test Results:

3.4 mvn -Dtest=${TestCi*le} test

We can use wildcards to match the test class to be executed. For example, here I use wildcards to specify the test class MavenDemo1Testand MavenDemo2Test:

$ mvn -Dtest=MavenDemo*Test test
 

Test Results:

3.5 mvn -Dtest=${TestCi*le},${TestClass} test

Of course, a combination of the above test mode is also possible, for example where the use of wildcards performed MavenDemo1Testand MavenDemo2Testthen specify precisely performed MavenPackageDemoTest:

$ mvn -Dtest=MavenDemo*Test,MavenPackageDemoTest test
 

Test Results:

4. Specify the test method

4.1 mvn -Dtest=TestCircle#mytest test

In addition to performing the test specified class, you can specify test methods, and between a #separated this example, I want to specify the execution MavenDemo1Testof test2the method:

$ mvn -Dtest=MavenDemo1Test#test2 test
 

Test Results:

4.2 mvn -Dtest=TestCircle#test* test

Methods can still use wildcards. For example MavenPackageDemoTest, the method to be executed here MavenPackageTestcan be matched to end with Test:

$ mvn -Dtest=MavenPackageDemoTest#*Test test
 

Test Results:

4.3 mvn -Dtest=TestCircle#testOne+testTwo test

mvn can also specify multiple methods under a certain test class +, separated by numbers, for example, specify MavenDemo1Testthe method test1()and t est2().

$ mvn -Dtest=MavenDemo1Test#test1+test2 test
 

5. Retry on failure

mvn also provides failure retries, set the number of failed retries, when the use case is executed until the use case succeeds or the number of failures reaches the upper limit, the command is as follows:

mvn -Dsurefire.rerunFailingTestsCount=2 test
 

Unfortunately, this function only supports Junit4. The instructions given on the official website are as follows:

6. Suite operation

In Junit5, we can actually use the @RunWith(JUnitPlatform.class)running test suite. For details, please refer to the article:
Introduction, composition, new features and basic usage of Junit5-common annotations, suite execution. However, when we use mvn to execute the suite: Suite:

import MavenTestPackage1.MavenPackageDemoTest;
import org.junit.platform.runner.JUnitPlatform;
import org.junit.platform.suite.api.SelectClasses;
import org.junit.platform.suite.api.SuiteDisplayName;
import org.junit.runner.RunWith;

@RunWith(JUnitPlatform.class)
@SuiteDisplayName(" ")
@SelectClasses({
        MavenPackageDemoTest.class,MavenDemo1Test.class
}
)
public class TestSuite {

}
 

mvn command:

mvn -Dtest=TestSuite test
 

The following error will appear:

It is said that it is because of the version of the surefire plug-in. When I switched to version 2.19, the suite was executed successfully, but the mvn test command failed again. Obviously this was not the result we wanted, so I started the road to Google. . . Finally found a relatively reasonable explanation! Sorry, let s be disappointed again, the explanation from Google to others is as follows (translated already~):
Simply means that the surefireplug-in is not yet supported Junit5under the succession to the implementation Junit4of @RunWith, they still need caring people come out to solve this problem had got ~~

6. Specify the default execution in Pom

In addition to using the command line, we can also specify the classes that we need to execute by default or the classes that do not need to be executed in the pom

For example, here I only want to execute by default MavenPackageDemoTest, and MavenDemoI don't want to execute the other two test classes at the beginning. At this time, I can pomdo the following configuration in and surefireadd under the plugin configuration:

<plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-surefire-plugin</artifactId>
   <version>3.0.0-M3</version>
   <configuration>
       <includes>
           <include>MavenPackageDemoTest.java</include>
       </includes>
       <excludes>
           <exclude>MavenDemo*.java</exclude>
       </excludes>
   </configuration>
</plugin>
 

Execute mvn testand view the results:

The results can be seen only from the execution of our default specified MavenPackageDemoTestin case, and the rest to MavenDemothe beginning of the test classes were not being executed.

7. Write at the end

Here maven-surefire-pluginare some common usages of plugins in test execution, as well as many other usages, such as:

  • Use very complex format matching
  • Control multi-threaded and multi-process execution at different levels

The follow-up will gradually study and supplement. If you are interested, you can directly refer to the official website to learn: