Triquetrum/Ptolemy/osgi-2-0

This page is notes about how the Ptolemy II osgi-2-0 repository that contains the Ptolemy II sources used by Triquetrum was set up. It is probably not that useful to most people.

1 See Also
2 Determining if there are any changes in the ptII tree to be ported to the osgi-2-0 tree
3 Creating a new Ptolemy osgi-2-0 p2 repository
4 Setting up a nightly build
- 4.1 Creating pom files
- 4.2 Running mvn
5 The Ptolemy II osgi-2-0 p2 repository
- 5.1 Setting up the p2 repository
6 Signing the Ptolemy II osgi-2-0 Jar Files
7 Adding the source files to the p2 repo

Determining if there are any changes in the ptII tree to be ported to the osgi-2-0 tree

To determine the date the last changes from the ptII tree were ported to the osgi-2-0 tree, see https://github.com/icyphy/ptII/branches to determine the last branch submitted for review. In August, 2020, it was origin/osgi-2-0-cq-2. https://github.com/icyphy/ptII/commits/origin/osgi-2-0-cq-2 says the last commit was on August 13, 2019, when code was merged from origin/osgi-2-0. https://github.com/icyphy/ptII/commits/origin/osgi-2-0 indicates that the last commit was August 9, 2019. So, we want to find any changes since August 10, 2019.

(There is probably a better way to do this)

Check out the ptII tree:

 git clone https://github.com/icyphy/ptII

Get the number of commits since a certain date:

 git log --since=2019-08-10 | egrep "^commit" | wc -l

Get the names f the java files that have changed since that many commits:

 git diff --name-only HEAD~95 HEAD~1 | egrep '.java$'

Creating a new Ptolemy osgi-2-0 p2 repository

If there are changes to the osgi-2-0 tree, then the p2 repository will need to be updated.

1) Check out the ptII osgi-2-0 branch

2) Build the repository:

mvn verify

3) Get the Ptolemy II code signing certificate, which requires access to a private repo. Contact Edward or Christopher for access. See Java Code Signing Certificates For Berkeley for details. Place the password in ~/.certpw and make that file readable only by yourself and root:

 chmod 0400 ~/.certpw

4) Create ~/.m2/settings.xml and update as necessary:

<settings>
  <profiles>
    <profile>
      <id>codesigning</id>
      <properties>

        <!-- See https://wiki.eecs.berkeley.edu/ptexternal/Main/JavaCodeSigningCertificatesForBerkeley for access to the password -->
        <keystore.path>/Users/cxh/src/ealprivi/passwords/ptkeystore.jks</keystore.path>

        <!-- Use 'keytool -list -keystore ptkeystore.jks -storepass xxxx' to get the alias. -->
        <keystore.alias>ptolemy</keystore.alias>
        <keystore.type>JKS</keystore.type>

        <!-- See https://wiki.eecs.berkeley.edu/ptexternal/Main/JavaCodeSigningCertificatesForBerkeley for access to the password -->
        <keystore.store.password>${keystore.password}</keystore.store.password>
        <keystore.key.password>${keystore.password}</keystore.key.password>
      </properties>
    </profile>
  </profiles>
</settings>

5) To sign the jars:

 mvn -Pcodesigning -Dkeystore.password=`cat ~/.certpw` package

This uses the Ptolemy II code signing certificate, see above for access to the keystore and password.

To verify that a jar is signed:

 jarsigner -verify -verbose -certs ./releng/org.ptolemy.tycho.update/target/repository/plugins/ptolemy.moml_*.jar

6) The p2 repository is hosted on the Berkeley Ptolemy webserver. See Ptolemy Website -> How To Log On to the webserver. The issue is that one cannot log on directly to calweb-basic-prod-01.ist.berkeley.edu, one must have a ssh key and transfer files. For any of this to work, one must have an account on login.eecs.berkeley.edu or swarmnuc2008.eecs.berkeley.edu and the /users/ptolemy/.ssh file on calweb-basic-prod-01.ist.berkeley.edu must be updated with your .ssh/id_rsa.pub key on login.eecs or swarmnuc2008.eecs. Once the keys are set up, transfer the repository:

cd releng/org.ptolemy.tycho.update/target/
tar -zcf /tmp/repo.tar.gz repository/
scp /tmp/repo.tar.gz swarmnuc2008.eecs.berkeley.edu:/tmp

Then log on to swarmnuc2008 and transfer the file:

ssh swarmnuc2008.eecs.berkeley.edu
scp /tmp/repo.tar.gz   ptolemy@calweb-basic-prod-01.ist.berkeley.edu:/tmp

7) Then, from login.eecs.berkeley.edu or swarmnuc2008.eecs.berkeley.edu, log on to the webserver:

ssh -l ptolemy calweb-basic-prod-01.ist.berkeley.edu
cd /users/ptolemy/apache2/ptweb/projects/chess/triq/p2
mv osgi-2-0-2 osgi-2-0-2.old
tar -zxf /tmp/repo.tar.gz
mv repository osgi-2-0-2

Setting up a nightly build

We would like to set up a nightly build, so we want to run Maven, create jar files and make them available somehow.

See Check out the ptII osgi-2-0 branch for getting access to the Ptolemy II osgi-2-0 repository

Creating pom files

POM-less Tycho Builds suggests creating osgi-2-0/.mvn/extensions.xml:

<?xml version="1.0" encoding="UTF-8"?>
<extensions>
  <extension>
    <groupId>org.eclipse.tycho.extras</groupId>
    <artifactId>tycho-pomless</artifactId>
    <version>0.24.0</version>
  </extension>
</extensions>

The Eclipse Tycho Tutorial suggests:

 cd osgi-2-0
 mvn org.eclipse.tycho:tycho-pomgenerator-plugin:generate-poms -DgroupId=ptolemy

This creates osgi-2-0/pom.xml and other */pom.xml files.

We then remove the */pom.xml files that were created:

 rm */pom.xml

Running mvn

The Eclipse Tycho Tutorial suggests running

 mvn clean verify

The tutorial states:

"Tip: It is not recommended to use the mvn clean install command, instead you should use mvn clean verify. If you would use the install option, the build result would be installed in your local Maven repository and this can lead to build problems.

The tutorial states:

"Warning If you try to execute this POM file, the build fail because you have not yet configured your dependencies."

mvn clean fails

If we run

mvn clean

we get

[ERROR] Cannot resolve project dependencies:
[ERROR]   Software being installed: diva 11.0.0.qualifier
[ERROR]   Missing requirement: diva 11.0.0.qualifier requires 'package org.w3c.dom.svg 1.1.0' but it could not be found
[ERROR]
[ERROR] See http://wiki.eclipse.org/Tycho/Dependency_Resolution_Troubleshooting for help.
[ERROR] Cannot resolve dependencies of MavenProject: ptolemy:diva:11.0.0-SNAPSHOT @ /Users/cxh/src/osgi-2-0/diva/.polyglot.build.properties: See log for details -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MavenExecutionException

So the next step is to fix the dependencies.

Fixing a dependency problem

Looking at Understanding Missing Requirement Messages and Using p2 update sites for dependencies resolutionUsing p2 update sites for dependencies resolution, the solution is to add the following to the osgi-2-0/pom.xml file:

  <repositories>
    <repository>
      <id>eclipse-mars</id>
      <url>http://download.eclipse.org/releases/mars</url>
      <layout>p2</layout>
    </repository>
  </repositories>

After that

 mvn clean

works

mvn verify fails

However,

 mvn verify

fails:

[INFO] Building diva 11.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[WARNING] The POM for org.w3c.dom.svg:svg-dom-java:jar:1.1.0 is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] osgi-2-0 ........................................... SUCCESS [  0.002 s]
[INFO] diva ............................................... FAILURE [  0.058 s]
[INFO] org.ptolemy.testsupport ............................ SKIPPED
[INFO] org.ptolemy.commons ................................ SKIPPED
[INFO] ptolemy.core ....................................... SKIPPED
[INFO] ptolemy.moml ....................................... SKIPPED
[INFO] ptolemy.actor.lib .................................. SKIPPED
[INFO] ptolemy.domains.sdf ................................ SKIPPED
[INFO] org.ptolemy.actor.gui.test ......................... SKIPPED
[INFO] ptolemy.domains.process ............................ SKIPPED
[INFO] org.ptolemy.core.test .............................. SKIPPED
[INFO] org.ptolemy.core.test.actor ........................ SKIPPED
[INFO] ptolemy.actor.lib.gui .............................. SKIPPED
[INFO] ptolemy.gui ........................................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9.218 s
[INFO] Finished at: 2016-06-28T16:32:25-07:00
[INFO] Final Memory: 92M/504M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project diva: Could not resolve dependencies for project ptolemy:diva:eclipse-plugin:11.0.0-SNAPSHOT: Failure to find org.w3c.dom.svg:svg-dom-java:jar:1.1.0 in https://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :diva

The fix was to remove the following from osgi-2-0/pom.xml:

  <dependencies>
    <!-- https://mvnrepository.com/artifact/org.axsl.org.w3c.dom.svg/svg-dom-java -->
    <dependency>
      <groupId>org.w3c.dom.svg</groupId>
      <artifactId>svg-dom-java</artifactId>
      <version>1.1.0</version>
    </dependency>
  </dependencies>

mvn verify fails because of GenerateHTMLTest

Running

 mvn verify

fails:

[ERROR] Failed to execute goal org.eclipse.tycho:tycho-compiler-plugin:0.24.0:compile (
default-compile) on project org.ptolemy.actor.gui.test: Compilation failure: Compilation failure:
[ERROR] /Users/cxh/src/osgi-2-0/org.ptolemy.actor.gui.test/src/test/java/org/ptolemy/actor/gui/test/GenerateHTMLTest.java:[39]
[ERROR] import ptolemy.actor.gui.GenerateCopyrights;
[ERROR] ^^^^^^^^^^^^^^^^^
[ERROR] The import ptolemy.actor.gui cannot be resolved
[ERROR] /Users/cxh/src/osgi-2-0/org.ptolemy.actor.gui.test/src/test/java/org/ptolemy/actor/gui/test/GenerateHTMLTest.java:[59]
[ERROR] System.out.println(GenerateCopyrights.generateHTML(null));
[ERROR] ^^^^^^^^^^^^^^^^^^
[ERROR] GenerateCopyrights cannot be resolved
[ERROR] 2 problems (2 errors)

A workaround:

 rm org.ptolemy.actor.gui.test/src/test/java/org/ptolemy/actor/gui/test/GenerateHTMLTest.java

However it seems like we don't have the dependencies properly set up.

Editing osgi-2-0/org.ptolemy.actor.gui.test/META-INF/MANIFEST.MF and adding a trailing comma and

  ptolemy.gui;bundle-version="10.1.0"

so that we have

Require-Bundle: org.ptolemy.testsupport;bundle-version="10.1.0",
 ptolemy.core;bundle-version="10.1.0",
 ptolemy.actor.lib;bundle-version="10.1.0",
 ptolemy.domains.sdf;bundle-version="10.1.0",
 ptolemy.gui;bundle-version="10.1.0"

IconAttribute

Running:

 mvn verify

fails with:

[ERROR] Failed to execute goal org.eclipse.tycho:tycho-compiler-plugin:0.24.0:compile (
default-compile) on project ptolemy.gui: Compilation failure: Compilation failure:
[ERROR] /Users/cxh/src/osgi-2-0/ptolemy.gui/src/ptolemy/vergil/icon/EditorIcon.java:[52]
[ERROR] import ptolemy.kernel.util.IconAttribute;
[ERROR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ERROR] Access restriction: The type 'IconAttribute' is not API (restriction on classpath entry '/Users/cxh/src/osgi-2-0/ptolemy.core/target/classes')

Access restriction on class due to restriction on required library rt.jar? (Stackoverflow) refers to What is the latest on "Access restriction" compilation errors, which suggests adding:

<plugin>
    <groupId>org.eclipse.tycho</groupId>
    <artifactId>tycho-compiler-plugin</artifactId>
    <version>${tycho.version}</version>
    <configuration>
        <compilerArgument>-warn:+discouraged,forbidden</compilerArgument>
    </configuration>
</plugin>

The ${tycho.version} was causing problems, so at the top of pom.xml we added:

  <properties>
    <target-version>0.1.0-SNAPSHOT</target-version>
    <tycho-version>0.24.0</tycho-version>
  </properties>

and then added the following to the plugins section

      <!-- See http://dev.eclipse.org/mhonarc/lists/tycho-user/msg03670.html -->
      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>tycho-compiler-plugin</artifactId>
        <version>${tycho-version}</version>
        <configuration>
          <compilerArgument>-warn:+discouraged,forbidden</compilerArgument>
        </configuration>
      </plugin>

Now

 mvn verify

works!

No explicit target runtime environment configuration

Running

 mvn verify

produces

[WARNING] No explicit target runtime environment configuration. Build is platform dependent.

So we add the following:

      <!-- This configures the target platform and the supported architectures. -->
      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>target-platform-configuration</artifactId>
        <version>${tycho-version}</version>
        <configuration>
          <environments>
            <environment>
              <os>linux</os>
              <ws>gtk</ws>
              <arch>x86</arch>
            </environment>
            <environment>
              <os>linux</os>
              <ws>gtk</ws>
              <arch>x86_64</arch>
            </environment>
            <environment>
              <os>win32</os>
              <ws>win32</ws>
              <arch>x86</arch>
            </environment>
            <environment>
              <os>win32</os>
              <ws>win32</ws>
              <arch>x86_64</arch>
            </environment>
            <environment>
              <os>macosx</os>
              <ws>cocoa</ws>
              <arch>x86_64</arch>
            </environment>
          </environments>
        </configuration>
      </plugin>

See the triquetrum pom.xml for an additional target section that could go in this plugin:

        <target>
          <artifact>
            <groupId>triquetrum</groupId>
            <artifactId>org.eclipse.triquetrum.target.platform</artifactId>
            <version>0.1.0-SNAPSHOT</version>
          </artifact>
        </target>

Clearly, the artifactId would need to be changed.

The Ptolemy II osgi-2-0 p2 repository

The Ptolemy II osgi-2-0 p2 repository is at https://ptolemy.berkeley.edu/projects/chess/triq/p2/

Setting up the p2 repository

We want to create a p2 repository for these files, see Bug #114: Create a p2 site for the Ptolemy II bundles]

This was a bit tricky. Follow the Eclipse Tycho Tutorial

I followed the releng/ and features/ subdirectory structure proposed by the tutorial, but this might not be necessary.

Below are the steps that were used to set up the releng/ and features/ directories. These steps are already done and the results are checked in to the osgi-2-0 repo.

Checked out the repo with: svn co https://repo.eecs.berkeley.edu/svn/projects/eal/ptII/branches/osgi-2-0
mkdir -p osgi-2-0/releng/org.ptolemy.tycho.update
Create the org.ptolemy.tycho.update project in the above directory
Create osgi-2-0/releng/pom.xml
mkdir -p osgi-2-0/features/org.ptolemy.tycho.feature
Create the org.ptolemy.tycho.feature project in the above directory
Create osgi-2-0/features/pom.xml
Running "mvn verify" creates ./releng/org.ptolemy.tycho.update/target/repository/plugins/diva_11.0.0.201606291522.jar

In the osgi-2-0 Jenkins build, one of the steps publishes artifacts to a SSH site on moog. The contents of releng/org.ptolemy.tycho.update/target/** are copied to ~www/cvswww/chess.eecs.berkeley.edu/triq/. A symbolic link from ~www/cvswww/chess.eecs.berkeley.edu/triq/p2/osgi-2-0 to ../releng/org.ptolemy.tycho.update/target/repository was created.

Signing the Ptolemy II osgi-2-0 Jar Files

Bug #178 Sign the Ptolemy Plug-ins wanted the Ptolemy II files in https://chess.eecs.berkeley.edu/ptexternal/src/p2/osgi-2-0/ to be signed.

To do this, Apache Maven Jarsigner Plugin suggested how to edit pom.xml:

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jarsigner-plugin</artifactId>
        <version>1.4</version>
        <executions>
          <execution>
            <id>sign</id>
            <goals>
              <goal>sign</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <!-- storetype etc. are defined in ~/.m2/settings.xml.                                       
               See http://stackoverflow.com/questions/3598424/jar-signing-strategy-in-maven-projects   
               mvn -Pcodesigning -Dkeystore.password=`cat ~/.certpw` package                           
          -->
          <storetype>${keystore.type}</storetype>
          <!-- See /users/ptII/adm/certs/ptkeystore on terra.eecs.berkeley.edu -->
          <keystore>${keystore.path}</keystore>
          <!-- Use 'keytool -list -keystore ptkeystore -storepass xxxx' to get the alias. -->
          <alias>${keystore.alias}</alias>
          <!-- See ~jenkins/.certpw on terra.eecs.berkeley.edu -->
          <storepass>${keystore.store.password}</storepass>
          <keypass>${keystore.key.password}</keypass>
        </configuration>
      </plugin>

http://stackoverflow.com/questions/3598424/jar-signing-strategy-in-maven-projects suggested using a profile and variables:

The ~/.m2/settings.xml file:

<settings>                                                                                             
  <profiles>
    <profile>
      <id>codesigning</id>
      <properties>
          <!-- See /users/ptII/adm/certs/ptkeystore on terra.eecs.berkeley.edu -->
        <keystore.path>/Users/cxh/txt/ptkeystore</keystore.path>
        <!-- Use 'keytool -list -keystore ptkeystore -storepass xxxx' to get the alias. -->
        <keystore.alias>ptolemy</keystore.alias>
        <keystore.type>JKS</keystore.type>
        <!-- See ~jenkins/.certpw on terra.eecs.berkeley.edu -->
        <keystore.store.password>${keystore.password}</keystore.store.password>
        <keystore.key.password>${keystore.password}</keystore.key.password>
      </properties>
    </profile>
  </profiles
</settings>

To sign the jars:

 mvn -Pcodesigning -Dkeystore.password=`cat ~/.certpw` package

This uses the Ptolemy code signing certificate, which is found on moog.eecs.berkeley.edu in /users/ptII/adm/certs/ptKeystore. The password is found in /root/.certpw

To verify that a jar is signed:

 jarsigner -verify -verbose -certs ./releng/org.ptolemy.tycho.update/target/repository/plugins/ptolemy.moml*.jar

Adding the source files to the p2 repo

The p2 site for the first cq may be found at http://chess.eecs.berkeley.edu/triq/p2/osgi-2-0-cq-1

To create this:

1) osgi-2-0-cq-1/pom.xml was updated

      <!-- To create source bundles, run mvn org.eclipse.tycho:tycho-source-plugin:plugin-source       
           See https://eclipse.org/tycho/sitedocs/tycho-source-plugin/plugin-info.html                 
        -->
      <plugin>
        <groupId>org.eclipse.tycho</groupId>
        <artifactId>tycho-source-plugin</artifactId>
	<version>${tycho-version}</version>
	<executions>
          <execution>
            <id>plugin-source</id>
            <goals>
              <goal>plugin-source</goal>
            </goals>
          </execution>
	</executions>
      </plugin>

      <!-- enable source feature generation -->
      <plugin>
	<groupId>org.eclipse.tycho.extras</groupId>
	<artifactId>tycho-source-feature-plugin</artifactId>
	<version>${tycho.extras.version}</version>

	<executions>
          <execution>
            <id>source-feature</id>
            <phase>package</phase>
            <goals>
              <goal>source-feature</goal>
            </goals>
          </execution>
	</executions>
      </plugin>

      <plugin>
	<groupId>org.eclipse.tycho</groupId>
	<artifactId>tycho-p2-plugin</artifactId>
	<version>${tycho-version}</version>
	<executions>
          <execution>
            <id>attached-p2-metadata</id>
            <phase>package</phase>
            <goals>
              <goal>p2-metadata</goal>
            </goals>
          </executions>
	</executions>
      </plugin>

See Creating a new Ptolemy osgi-2-0 p2 repository

Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Triquetrum/Ptolemy/osgi-2-0

Contents

See Also

Determining if there are any changes in the ptII tree to be ported to the osgi-2-0 tree

Creating a new Ptolemy osgi-2-0 p2 repository

Setting up a nightly build

Creating pom files

Running mvn

mvn clean fails

Fixing a dependency problem

mvn verify fails

mvn verify fails because of GenerateHTMLTest

IconAttribute

No explicit target runtime environment configuration

The Ptolemy II osgi-2-0 p2 repository

Setting up the p2 repository

Signing the Ptolemy II osgi-2-0 Jar Files

Adding the source files to the p2 repo

Breadcrumbs

Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Triquetrum/Ptolemy/osgi-2-0

Contents

See Also

Determining if there are any changes in the ptII tree to be ported to the osgi-2-0 tree

Creating a new Ptolemy osgi-2-0 p2 repository

Setting up a nightly build

Creating pom files

Running mvn

mvn clean fails

Fixing a dependency problem

mvn verify fails

mvn verify fails because of GenerateHTMLTest

IconAttribute

No explicit target runtime environment configuration

The Ptolemy II osgi-2-0 p2 repository

Setting up the p2 repository

Signing the Ptolemy II osgi-2-0 Jar Files

Adding the source files to the p2 repo