OCL/Dev/Releng/Issues Documentation

This wiki entry has been created in order to document specific releng-related issues, final diagnosis, and how they were solved.

Tycho

Secret debugging information may be found in .../target/work/data/.metadata/.log

A workspace refresh may be needed to see all files such as .../target/work/data/.metadata/trec.log

Tycho tests fail with ResourceException

Two Completion proposal tests failed with

 org.eclipse.core.internal.resources.ResourceException: Errors occurred during the build.
 at org.eclipse.core.internal.resources.Project$1.run(Project.java:568)
 at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2240)
 at org.eclipse.core.internal.resources.Project.internalBuild(Project.java:544)
 at org.eclipse.core.internal.resources.Project.build(Project.java:112)

and a debugger test launch failed with a null return from

 launch = consolePage.launchDebugger();
 assertNotNull(launch);

Debugging revealed that the problem was that many platform services had not started up because the PlatformAdmin provided by org.eclipse.osgi.compatibility.state was not active. Attempts to 'install' the org.eclipse.osgi.compatibility.state fragment failed and no help was forthcoming from the tycho-user mailing list so the tests were just disabled[1].

Solution: Contributed as [2]

In the end this only fixed one out of two Completion tests, but the residual failure was a more plausible path problem. The log message was cured.

The debugger test was a red herring; it had never worked. [3]

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=526252

[2] https://wiki.eclipse.org/Tycho/FAQ#How_do_I_install_the_org.eclipse.osgi.compatibility.state_fragment.3F

[3] https://bugs.eclipse.org/bugs/show_bug.cgi?id=529886

All Tycho surefire tests pass, but tests fail

Symptom The tycho-surefire-plugin summary shows N out of N tests ran and passed with zero failures, but the build terminates.

 [ERROR] Failed to execute goal org.eclipse.tycho:tycho-surefire-plugin:1.0.0:test (default-test) on project org.eclipse.ocl.examples.xtext.tests:
 An unexpected error occured while launching the test runtime (return code -1073740771). See log for details. -> [Help 1]

Clearly bad but not very diagnostic. -1073740771, is probably 0xc0020001 User Callback Exception after application Close. One might suspect that workspace shutdown failed, perhaps because of an obstinate worked thread such as a builder.

Initial debugging pointed at org.eclipse.osgi.compatibility.state which was fixed and at unhelpful new "Do you want to convert to OCL project" dialogs which were also fixed, or rather converted to IllegalStateException in test environments.

The test remained obstinately intermittent and actually hysterical. The same application code could consistently pass if the application code had evolved by successive good additions to a good state, but consistently fail after evolution by successive removal of code from a bad state. Consequently many erroneous hypotheses could be pursued until suddenly the hysteresis flipped. The lack of primary debug evidence leaves all prevailing hypotheses as just guesses.

Suspicion is that the hysteresis is caused by an old Tycho build not being terminated and so locking filrs that then cause an othwise good new build to appear bad.

Raised as Tycho bugs [1] and [2], OCL bug is: [3].

Solution: Not resolved yet.

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=529928

[2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=529929

[3] https://bugs.eclipse.org/bugs/show_bug.cgi?id=529797

The Tycho surefire test failures are missing

Symptom Failed to execute goal org.eclipse.tycho:tycho-surefire-plugin:1.1.0:test (default-test) on project org.eclipse.qvtd.xtext.qvtbase.tests: There are test failures.

But there are no test failures.

Further clue Please refer to /jobs/genie.qvtd/qvtd-branch-tests/workspace/tests/org.eclipse.qvtd.xtext.qvtbase.tests/target/surefire-reports/plugin for the individual test results.

Where .../plugin.log contains multiple failures the first of which is: org.eclipse.e4.core.di.InjectionException: Unable to process "CommandProcessingAddon.broker": no actual value was found for the argument "IEventBroker".

This is https://bugs.eclipse.org/bugs/show_bug.cgi?id=538564 which will eventually be fixed by Tycho 1.3.0, but there may be a transition period while an org.eclipse.equinox.ds dependency is needed. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=538600

The Tycho surefire log file and test failures are missing

Symptom Failed to execute goal org.eclipse.tycho:tycho-surefire-plugin:1.2.0:test (default-test) on project org.eclipse.ocl.examples.xtext.tests: An unexpected error occured while launching the test runtime (return code 137). See log /jobs/genie.ocl/ocl-branch-tests/workspace/tests/org.eclipse.ocl.examples.xtext.tests/target/work/data/.metadata/.log for details.

But the .log file does not exist (and there are no test results). From the elapsed time it looks very much as if both tycho and maven surefire ran, but it might have been just maven on a slow day.

Rerunning apparently gave the same result, but no, there was a genuine test failure on the rerun that made the superficial tycho failure analysis look the same, except the test results were there to explain.

Conclusion. Tycho testing is not reliable. Do not waste time investigating a failure until it is definitely reproducible.

Missing SWT classes

Symptom The type org.eclipse.swt.graphics.Image cannot be resolved

This occurs during Tycho compilations. The problem arises on 4.10 (2018-12) after removal of win32 x86 support. It is easily fixed by removing the root pom.xml definition that win32 x86 is a supported environment. https://bugs.eclipse.org/bugs/show_bug.cgi?id=539656 may eventually lead to a better diagnostic.

Jenkins build failure cannot be reproduced locally

First possibility: Using Maven/Tycho classes are built to a 'classes' rather than 'bin' tree consequently any reference to a 'bin' file will fail. The same reference locally may succeed, confusingly using a relic of a workspace build.

Second possibility: Jenkins is running on Linux whereas local builds are on Windows. Differences may arise through failure to use the correct path/file separators or non-canonical representations.

Third possibility: Inconsistent Java versions with magic new functionality such as modules.

Strange @NonNull related compiler failures on Java 11

Once the build migrates to Tycho 2.1.0 to accommodate the mandatory Java 11 usage, a few strange compiler failures appeared. These are all discussed in Bug 569379.

The theory that it was an ECJ wrt JDT divergence was investigated and rejected by

   <ecj-version>3.23.0</ecj-version>

       <plugin>
         <groupId>org.eclipse.tycho</groupId>
         <artifactId>tycho-compiler-plugin</artifactId>
         <version>${tycho-version}</version>
          <dependencies>
           <dependency>
             <groupId>org.eclipse.jdt</groupId>
             <artifactId>ecj</artifactId>
             <version>${ecj-version}</version>
           </dependency>
         </dependencies>

The correct spellings of ecj-version may be found at https://mvnrepository.com/artifact/org.eclipse.jdt/ecj. Since ECJ is in Maven central no special repo needs specifying.

It would appear that Tycho's auto-inference of modules that came in at Tycho 1.7.0 may be the problem and is fixed by:

       <plugin>
         <groupId>org.eclipse.tycho</groupId>
         <artifactId>tycho-compiler-plugin</artifactId>
         <version>${tycho-version}</version>
         <configuration>
           <deriveReleaseCompilerArgumentFromTargetLevel>false</deriveReleaseCompilerArgumentFromTargetLevel>
         </configuration>

It would also appear that Tycho passes the wrong org.eclipse.jdt.core.prefs so it may be necessary to ensure consistency for all projects.

Buckminster

No component named ... is known to Buckminster

Date: 01/02/2012.

Problem: Juno M5 S-build failed due to the following error:

INFO: perform '--properties' '/opt/users/hudsonbuild/workspace/buckminster-mdt-ocl-tools-3.2-master/buildroot/build.properties' '-Dbuckminster.reference.repository=https://hudson.eclipse.org/hudson/job/buckminster-mdt-ocl-tools-3.2-master/lastSuccessfulBuild/artifact/MDT-OCL.p2.repository/' 'org.eclipse.ocl.releng.tools.build:eclipse.feature#build'
No component named org.eclipse.ocl.doc.source:eclipse.feature/[3.1.0.v20111215-0955,3.1.0.v20111215-0955] is known to Buckminster

This only occurred with OCL Tools S-build. Nightly and Integration builds worked. OCL Core S-Build succed as well.

Diagnosis:

For S-builds, we are using our own miletones repository to feed our Tools components with the Core ones.

During this milestone development, doc (and other features) were "restructured", so that they were previously built as OCL Core components, and they are currently built as OCL Tools components. So, In a Tools build, they are supposed to be materialized into the workspace. However, after deeply analyzing the job's console and workspace, I've realized that these features are being materialized into the target platform.

My current rmap is configured so that every OCL feature/plugin from a provided repository (tipically, a repository with Core componentes), is installed in the target platform, and the remaining are supposed to be materialized into the workspace from our GIT repository.

For Nightly and Integration builds, the repository from our Core job is used so just the expected Core components are installed into the target platform, hence, the doc plugin and feature is materialized into the workspace, and no problem with sources features.

For Stable builds, our milestones repository is used. However, it's a composed repository which also composes repositories from previous releases. Unfortunately, this means that non-Core plugins and featres (for example, doc ones) are available (from a repository of a previous milestone) and so installed into the target platform. Obviously this is incorrect and provokesthat they can't be materialized into the workspace and therefore no source feature is generated for them.

Solution:

To solve this, I think that I'll definitelly specify in the tools.rmap a locator whose pattern binds to the specific set of features/plugins which are expected for Core components. So the change is the following:

Replace:

<rm:locator pattern="^org\.eclipse\.ocl.*$" searchPathRef="ocl-core" failOnError="false"/>

By:

<rm:locator pattern="^org\.eclipse\.ocl(\.(ecore|uml|all|core\.sdk))?(\.source)?$" searchPathRef="ocl-core"/>

Commit bfed05ba761eeaf6e090b6e0e86e096509c0545c

No suitable provider for component com.google.collect... was found in resourceMap file

Date: 06/02/2012

Problem: Tests job fails due to an unresolved com.google.collect bundle

ERROR [0087] : No suitable provider for component com.google.collect:osgi.bundle/1.0.0 was found in resourceMap file:/opt/users/hudsonbuild/workspace/buckminster-mdt-ocl-branch-tests/org.eclipse.ocl.git/releng/org.eclipse.ocl.releng.buckminster/releng/ocl-tools.rmap
ERROR [0087] : No suitable provider for component com.google.collect:osgi.bundle/1.0.0 was found in searchPath xtext
ERROR [0087] : Rejecting provider p2({0}/modeling/tmf/updates/nightly[file:/home/data/httpd/download.eclipse.org/modeling/tmf/updates/nightly]): No component match was found

Diagnosis:

Every com.google.* bundle is being resolved against the Xtext repository, which is redistributing such a bundles. Since Xtext is not using such a bundle (guava is used instead), they are not distributing it anymore. However, due to an acceleo dependency, our build requires com.google.collect bundle. This bundle is distributed in the Orbit repository.

Solution:

Narrowing which com.google* bundles are fetched from Xtext. In this case every bundle distributed by Xtext which is not present into the Orbit repository:

Replacing:

<rm:locator pattern="^com\.google.*" searchPathRef="xtext"/>

By:

<rm:locator pattern="^com\.google.(inject|guava)$" searchPathRef="xtext"/>

N.B: Another interesting question is why Xtext is distributing such a third party bundle and it's not in the Orbit repository. I believe that we should fetch these bundles from Orbit repository.

Commit 3d2aec2dc6fb7c0840fbe3d45428254e79e1f72d

No available bundle exports package 'org.apache.log4j'

Date: 23/10/2014

Problem: Tests job fails due to an unresolved org.apache.log4j bundle

Error: file /jobs/genie.modeling.mdt.ocl/buckminster-ocl-branch-tests/workspace/org.eclipse.ocl.git/examples/org.eclipse.ocl.examples.codegen.asm3/src/org/eclipse/ocl/examples/codegen/asm3/ASM3JavaAnnotationReader.java, line 21: The import org.apache cannot be resolved
Error: file /jobs/genie.modeling.mdt.ocl/buckminster-ocl-branch-tests/workspace/org.eclipse.ocl.git/examples/org.eclipse.ocl.examples.codegen.asm3/src/org/eclipse/ocl/examples/codegen/asm3/ASM3JavaAnnotationReader.java, line 38: Logger cannot be resolved to a type
Error: file /jobs/genie.modeling.mdt.ocl/buckminster-ocl-branch-tests/workspace/org.eclipse.ocl.git/examples/org.eclipse.ocl.examples.codegen.asm3/src/org/eclipse/ocl/examples/codegen/asm3/ASM3JavaAnnotationReader.java, line 38: Logger cannot be resolved
Error: file /jobs/genie.modeling.mdt.ocl/buckminster-ocl-branch-tests/workspace/org.eclipse.ocl.git/examples/org.eclipse.ocl.examples.codegen.asm3/src/org/eclipse/ocl/examples/codegen/asm3/ASM3JavaAnnotationReader.java, line 200: Logger cannot be resolved to a type
Error: file /jobs/genie.modeling.mdt.ocl/buckminster-ocl-branch-tests/workspace/org.eclipse.ocl.git/examples/org.eclipse.ocl.examples.codegen.asm3/META-INF/MANIFEST.MF, line 11: No available bundle exports package 'org.apache.log4j'

Diagnosis:

For the most part org.apache.log4j and other utility imports are imported as packages rather than required as bundles, avoiding any need to know which bundle provides them.

As a consequence of rearranging features, the examples.xtext.tests plugin that did have a require bundle was removed leading to the above error.

Solution:

The buckminster.cspex may have explicit bundle dependencies avoiding the need for artificial plugin declarations.

Adding:

<cs:dependency name="org.apache.log4j" componentType="osgi.bundle" />

ensures that Buckminster loads the bundle.

commit b4f4a4c01397bec6e345f8032b81061e8caa8e1a

which also uses Orbit directly now that everything is in Orbit.

Attempt to use an unresolved node.

Date: 01/02/2012.

Problem: Tests job failed due to the following error:

ERROR: Attempt to use an unresolved node. Request is org.eclipse.ocl.uml:osgi.bundle/[4.0.0,5.0.0)

org.eclipse.buckminster.core.metadata.model.UnresolvedNodeException: Attempt to use an unresolved node. Request is org.eclipse.ocl.uml:osgi.bundle/[4.0.0,5.0.0)

Diagnosis:

When solving the problem in comment 1, I naively assumed that when specifying a concrete set of CORE components to be fetched in a Tools build, if the component is not found in the ocl-core search path, such a build must fail. I completely forgot the tests build which also uses the ocl-tools cquery and rmap files. The "resolve.ocl.core" property guides the build to resolve, otherwise not resolve, the OCL Core components from our p2 repositories. Therefore, when solving Core components in such a Search Path, the failOnError must be false, so that the tests build may later fetch OCL Core components from our GIT repository and materialize them into the job's workspace.

Solution:

Reinstate the failOnError = false property into the ocl-tools.rmap:

<rm:locator pattern="^org\.eclipse\.ocl(\.(ecore|uml|all|core\.sdk))?(\.source)?$" searchPathRef="ocl-core" failOnError="false"/>

Commit 3d2aec2dc6fb7c0840fbe3d45428254e79e1f72d

Packaging site

Date: 14/04/2012

Problem: Tests build failed due to an error [1] during the OCL SDK packaging. This error had previously appeared (Bug 363208 comment 2) and it's related to the process of packaging the zips, recently solved in that bug.

Diagnosis: The main inconvenience with the issue is that the Job's console doesn't provide any useful information about the problem. This can me investigated in local by adjusting the ANT logger level set by buckminster : Windows -> Preferences -> Buckminster -> Ant logger level = DEBUG. However in local the SDK zip packaging worked well so I can't reproduce the failed process in debug mode.

I've found a link[2] which shows how to change the ANT logger level for buckminster headless. Unfortunately, buckminster application commands are configured in hudson, vi its web UI (by the means of hudson-buckminster plugin) and there was not way to configure that in hudson web UI.

I moved the question to cross project [3] with no responses so far.

[1]

[ant] Packaging site: mdt-ocl-Update
INFO: [end org.eclipse.ocl.releng.tools.build:eclipse.feature$1.0.0.qualifier#package.mdt-ocl-Update]
INFO: [start org.eclipse.ocl.releng.tools.build:eclipse.feature$1.0.0.qualifier#site.tp]
INFO: [end org.eclipse.ocl.releng.tools.build:eclipse.feature$1.0.0.qualifier#site.tp]
INFO: [start org.eclipse.ocl.releng.tools.build:eclipse.feature$1.0.0.qualifier#package.mdt-ocl-SDK]
ERROR: org.eclipse.core.runtime.CoreException: <https://hudson.eclipse.org/hudson/job/buckminster-mdt-ocl-branch-tests/ws/org.eclipse.ocl.git/releng/org.eclipse.ocl.releng.tools.build-feature/packaging.ant>:208: Java returned: 13
org.eclipse.core.runtime.CoreException: <https://hudson.eclipse.org/hudson/job/buckminster-mdt-ocl-branch-tests/ws/org.eclipse.ocl.git/releng/org.eclipse.ocl.releng.tools.build-feature/packaging.ant>:208: Java returned: 13
at org.eclipse.buckminster.ant.AntRunner.handleInvocationTargetException(AntRunner.java:167)
at org.eclipse.buckminster.ant.AntRunner.run(AntRunner.java:322)
at org.eclipse.buckminster.ant.actor.AntActor.internalPerform(AntActor.java:254)
at org.eclipse.buckminster.core.actor.AbstractActor.perform(AbstractActor.java:195)
at org.eclipse.buckminster.core.internal.actor.PerformManager$DirectActionInvocation.execute(PerformManager.java:143)
at org.eclipse.buckminster.core.internal.actor.PerformManager.internalPerform(PerformManager.java:454)
at org.eclipse.buckminster.core.internal.actor.PerformManager.perform(PerformManager.java:293)
at org.eclipse.buckminster.core.internal.actor.PerformManager.perform(PerformManager.java:305)
at org.eclipse.buckminster.core.commands.Perform.internalRun(Perform.java:108)
at org.eclipse.buckminster.core.commands.WorkspaceCommand.run(WorkspaceCommand.java:91)
at org.eclipse.buckminster.cmdline.AbstractCommand.basicRun(AbstractCommand.java:200)
at

[2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=294440

[3] http://dev.eclipse.org/mhonarc/lists/cross-project-issues-dev/msg07517.html

Solution: Not resolved yet.

Similar problem occurred for QVTo. Went away after moving to separate HIPP.

The application could not start. Details can be found in the log.

This may occur when starting JUnit tests on Hudson.

See also [1].

In this situation you may found some log information in the workspace metada. In this case, you should look for a log file in the following location:

https://hudson.eclipse.org/hudson/job/buckminster-mdt-ocl-tools-kepler-master/ws/buildroot/buckminster.workspace/.metadata/.plugins/org.eclipse.pde.core/pde-junit/

One occasion the problem may have been caused by temporarily changing a launch configuration to use explicitly nominated plugins and then reverting. It may be that the obsolete but retained selections in the config cause trouble. This was cure by precisely reverting to the original launch configuration.

Export has failed because no test session is available

INFO:  junit '--launch' 'org.eclipse.ocl.examples.xtext.tests/.settings/org.eclipse.ocl.examples.xtext.tests (plugin).launch'
'--flatXML' '--maxTimeAwaitJunitReport' '60' '--output' '/home/hudson
/genie.modeling.mdt.ocl/.hudson/jobs/buckminster-ocl-branch-tests/workspace/MDT-OCL.test.results/org.eclipse.ocl.examples.xtext.plugin.tests.xml'
java.lang.IllegalArgumentException: Export has failed because no test session is available. (Maybe the argument -maxTimeAwaitJunitReport will help)

Originally happened Bug 371327 and suggestion was that a com.google.collect dependency might help. The problem was side-stepped by eliminating Acceleo from the builds by using dierect OCL to Java code generation.

Problem has reoccurred with the introduction of an org.eclipse.pde.core dependency.

Bug 251740 comment 40 also suggests that lack of precision on org.apache.commons deopendencies may leave Buckminster undecided about how to proceed.

-maxTimeAwaitJunitReport does not seem to help.

The above hang is diagnosable by looking in the Hudson workspace at .../ws/buildroot/junit-workspace/.metadata/.log where

org.osgi.framework.BundleException: Could not resolve module: org.eclipse.equinox.p2.publisher.eclipse [121]
  Unresolved requirement: Import-Package: org.eclipse.equinox.internal.frameworkadmin.equinox

had been present and ignored for some time. Addition of the new dependency caused the problem to occur in an important plugin and so caused the JUnit Eclipse launch to fail.

See Bug 432393

A similar problem occurred when a launch configuration used a non-ASCII character. In this case the failure occurred before the junit-workspace and its log was created. See Bug 473752

Problem occurred again at Oxygen M2 when SWT imposed a minimumn requirement of GTK 2.24 which was not available on Hudson. Once again the main console log just reported no test session. The junit-workspace/.metadata/.log identified

java.lang.UnsatisfiedLinkError: Could not load SWT library. Reasons:
 /jobs/genie.ocl/ocl-oxygen-master/workspace/buildroot/buckminster.workspace/.metadata/.plugins/org.eclipse.pde.core/pde-junit/org.eclipse.osgi/224/0/.cp/libswt-pi-gtk-4712.so: /lib64/libc.so.6:
version `GLIBC_2.14' not found (required by /jobs/genie.ocl/ocl-oxygen-master/workspace/buildroot/buckminster.workspace/.metadata/.plugins/org.eclipse.pde.core/pde-junit/org.eclipse.osgi/224/0/.cp/libswt-pi-gtk-4712.so)
no swt-pi-gtk in java.library.path
/opt/public/hipp/homes/genie.ocl/.swt/lib/linux/x86_64/libswt-pi-gtk-4712.so: /lib64/libc.so.6:
version `GLIBC_2.14' not found (required by /opt/public/hipp/homes/genie.ocl/.swt/lib/linux/x86_64/libswt-pi-gtk-4712.so)
Can't load library: /opt/public/hipp/homes/genie.ocl/.swt/lib/linux/x86_64/libswt-pi-gtk.so

The eventual fix, see Bug 510583, Bug 498429 required an update of Hudson to SLES12, change of Hudson XVNC settings and replacement of metacity by icewm. Replacement of metacity by mutter just led to a hang.

A similar (test start but no log) problem was perhaps due to a missing org.apache.batik.css version in Orbit; see Bug 522740. Inconsistency only became apparent once *.rmap providers were sufficiently limited to give an early no provider build failure.

Invalid signature file digest for Manifest main attributes

Date: 23/10/2014.

Problem: Build terminates abruptly after signing completes

[ant] Waiting for signing to complete. This may take more then 60 minutes
[ant] Obtaining signed file from staging area
INFO:  [end org.eclipse.ocl.releng.build:eclipse.feature$1.3.0.qualifier#site.signed]
INFO:  [start org.eclipse.ocl.releng.build:eclipse.feature$1.3.0.qualifier#site.packed]
java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
Caused by: Terminating xvnc.
Archiving artifacts

There appears to be no clue as to which Manifest might be in error.

Diagnosis:

This only occurs for signed builds, so takes a long time to reach the problem. It arose once test features were added to the P2 repo.

Presumably something coming back from the signing process is unhappy. Googling suggests that extra already signed files in META-INF can be bad.

One early problem was a MANIFEST.MF that had a "Bundle-ManifestVersion: 2" line but no "Manifest-Version: 1.0" line.

Later on the problem seemed to be a lazy practice of using tests features to import required bundles. No problem until the true packaging purpose was used. Then unwanted plugins such as "org.eclipse.test" try to get packaged and (hypothesis) since "org.eclipse.test" is not packed it fails to copy across to the repo adequately and the signer/subsequent checker barfs.

Solution:

Only include plugins/features to be packaged within features.

Use the Buckminster *.cspex file to define dependencies for plugins/features to be loaded but not packaged.