Tycho/Testing with Surefire

Maven doc is available at: http://www.eclipse.org/tycho/sitedocs/tycho-surefire/tycho-surefire-plugin/test-mojo.html . But this page will cover concepts and advanced features of the tycho-surefire-plugin:

Dependency resolution and Test Runtime

In order to execute tests, Tycho needs an OSGi runtime which contains:

• the test bundle
• dependencies of the test bundle
• (optionally) dependencies added as dependencies in configuration of tycho-surefire-plugin
• Tycho test harness (org.eclipse.tycho.surefire.osgibooter) and its dependencies

There are various strategies possible for creating the OSGi runtime. They are controlled by the parameter testRuntime of tycho-surefire-plugin. For easiest and maximal benefit of testing during build, it's recommended to not set this testRuntime property and leave it to default in the main tycho-surefire-plugin configuration that will run on your eclipse-test-plugin. Other strategies are aimed at allowing "batch" testing and can/should be configured into profiles dedicated to this use-case, not made default.

Default behaviour: Create OSGi runtime from target platform

By default, Tycho looks inside the target platform to resolve the dependencies, and creates an RCP application in ${project.build.directory}/work (except if you did override the work parameter of tycho-surefire-plugin). You can see the application definition in the work/config folder. The list of plugins used in the application can be read in the osgi.bundles property of the config.ini file.

This strategy guarantees that you run your application with the very minimal set of necessary plugins. It ensures better isolation for your test as it avoids side-effects from unnecessary plugins, It also helps to make your tests faster using the minimum number of bundles and referencing existing bundles on the filesystem rather than copying them for test runtime installation. This is why it is the recommended configuration for default tycho-surefire-plugin execution.

Note that this OSGi runtime is not installed using p2. So code that expects any RCP application to be p2-installed may fail with Tycho whereas it works in Eclipse. In such case, it is recommended that you inspect your code and find out whether you can replace usage of p2 to introspect your application by pure usage of OSGi Platform APIs.
Also, this application doesn't have the standard layout (with a plugins directory) so it may also confuse other pieces of code expecting a standard layout to work well. For example, it makes PDE have an empty target platform by default, leading to failed builds in test workspace: https://bugs.eclipse.org/bugs/show_bug.cgi?id=343156 . Recommended workaround is to make you test set up a good target platform.

p2Installed on provisioned RCP application

This mode allows you to run the test against an already provisioned p2-friendly RCP application. It is pretty useful as it allows to reuse the configuration of the test defined in your pom.xml, but against an application which is already existing. Use cases for this strategy can be:

• Testing against all the contents of the product you are developing, and see whether some side-effects prevent test from working
• Testing against an older/newer version of the product you are developing
• Testing against another product than the one you are developing

The important parameters to configure such a strategy:

• <testRuntime>p2Installed</testRuntime> tells surefire that it won't have to create an application, but instead to run inside the chosen application
• <profileName>...</profileName> is the name of the p2 profile of the application. You can read find the profiles installed in your application in path/to/application/p2/org.eclipse.equinox.p2.engine/profileRegistry/. Default profile can be found in configuration/config.ini as the value of the eclipse.p2.profile property. Some example of profileNames include: epp.package.rcp, epp.package.jee, jbds ...
• <work>/path/to/application</work> is the location of the application where to run tests. You can either change the value of this parameter, or decide to populate the default location (${project.build.directory}/work) with your application to test, by copying it or unzipping it there.

What happens in that case is that Tycho will ask p2 to install the necessary dependencies in your application. As it re-uses the p2 profile, it will only install the dependencies that are not already available in your test runtime. Other dependencies will be installed from the module target platform.

Here is an example of pom file using this

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <parent>
                <groupId>org.jboss.tools.ws</groupId>
                <artifactId>tests</artifactId>
                <version>1.5.0-SNAPSHOT</version>
        </parent>
        <groupId>org.jboss.tools.ws.tests</groupId>
        <artifactId>org.jboss.tools.ws.core.test</artifactId>
 
        <packaging>eclipse-test-plugin</packaging>
 
        <profiles>
                <profile>
                        <id>run-on-jbds</id>
                        <build>
                                <plugins>
                                        <plugin>
                                                <groupId>org.eclipse.tycho</groupId>
                                                <artifactId>tycho-surefire-plugin</artifactId>
                                                <version>${tychoVersion}</version>
                                                <configuration>
                                                        <testRuntime>p2Installed</testRuntime>
                                                        <work>/home/mistria/jbdevstudio-7.1.0.Alpha2/studio</work>
                                                        <profileName>jbds</profileName>
                                                </configuration>
                                        </plugin>
                                </plugins>
                        </build>
                </profile>
        </profiles>
</project>

With this example mvn clean verify -Prun-on-jbds will compile test against target platform, then install it inside profile jbds of application located at /home/mistria/jbdevstudio-7.1.0.Alpha2/studio and then run the test there. As the application already contains most of the dependencies for this test to run, they will be reused and no new one will get installed.

As this is quite a different behaviour from the default one and as it as strong assumptions on the pre-existence of the test runtime, it's definitely better to configure this in a profile, as you can still benefit from simplicity and efficiency of default behaviour.

p2Installed with reference to a p2 product IU

Tycho will use p2 to generate an application available in the target platform for the given profileName. This application will be provisioned in the location specified by work parameter. Then it will perform the steps explained above to install necessary bundle in this application and run tests.