PTP/release notes/2.0

What's new in PTP 2.0

See the new features for PTP Core and new features for PLDT as well as links to all the online help for PTP and PLDT so far.

Impatient and already know how to install? Go directly to the PTP 2.0 Update site

Prerequisites

• OS: Linux or Mac OS X 10.4.x or 10.5.x (for PTP Core only; PLDT should run anywhere Eclipse runs, including Windows.)
• Java 1.5 or later (Note: for Linux, must be Sun or IBM's. See below.)
• OpenMPI 1.2 (the earlier 1.0.2 version is also supported)
• Eclipse 3.3 (also known as Europa)
• CDT 4.0.2 or later
• gdb 6.3 - 6.8 on Linux and gdb 6.3 on MacOS X (earlier versions may also work)

Installation

You will need to install Java, OpenMPI, Eclipse, CDT (Eclipse C/C++ Development Tools), and PTP, then build a small platform-specific set of runtime components. The steps are detailed below:

Install Java

Java requirement: If you are using the Linux operating system, download and install either the Sun Java Runtime Environment or the IBM Java Runtime Environment. The version of Java that ships with Linux will not work with PTP. See the Trouble Shooting section for more information.

Install Open MPI

Download and unpack Open MPI 1.2. If you are using a pre-built binary for Open MPI (such as one that is installed as part of a Linux installation), then it must be a developer version. If there is no developer version available, then you will need to build from source instead. (Note: MAC OSX 10.5 - Leopard - contains OpenMPI but not the developer version. Be sure you build/install this, *and* put its location, e.g. /usr/local/bin ahead of /usr/bin)

Configure, build and install from source

In the main Open MPI source directory, run the configure command:

       ./configure --with-devel-headers

and any other options required for your installation.

Build and install Open MPI using:

       make install

(If you don't have write authority to where it will install, you may need to do sudo make install)

Note that if you install Open MPI in a nonstandard location, you may need to build PTP differently. See the troubleshooting section below. Make sure that your PATH and LD_LIBRARY_PATH point to appropriate places if necessary also.

Test your MPI installation by running a small MPI job, e.g.

       mpicc test.c -o test
       mpirun -np 4 test

Install Eclipse

Install Eclipse 3.3.x (SDK or Platform Runtime Binary), and CDT 4.0.2 or later, for your architecture. PTP Release 2.0 requires Eclipse 3.3.* (also known as Europa ) and CDT 4.0.2 or later.

Note that Since Eclipse 3.4 (Ganymede) has been released, you have to look a bit closer to get 3.3 (Europa). The Europa downloads are here.

There are three choices here:

1. Install Eclipse SDK - e.g. full version ("Classic") and CDT 4.0.2 or later - if you plan to do Java or Eclipse development in addition to C/C++, you'll need this.
2. (easiest, lightweight, smaller footprint) Install Eclipse IDE for C/C++ development ( available from Europa Winter maintenance release, includes CDT 4.0.3). - if you only plan to do C/C++ development, this is all you need. This is about 1/3 to 1/4 the size of the full SDK, and doesn't include the Java or Eclipse Plug-in development parts, which I assume you don't need. The UI is simpler since you don't have all those additional options.
3. (lightweight, smaller footprint, assemble it yourself) Install Eclipse Platform Runtime Binary and CDT 4.0.2 or later

(1) Full Eclipse install

1. Go to Eclipse downloads for 3.3.2 and download "Eclipse classic" for your platform. This is the full SDK.

(2) Lightweight install for just C/C++ development

1. Go to Eclipse downloads for Europa and download "Eclipse IDE for C/C++ Developers." This is the basics of the Eclipse Platform, plus CDT, already packaged together.

(3) Lightweight Platform plus CDT, Assemble it yourself

1. Go to Eclipse 3.3.2 release page and get the Platform runtime binary for your platform.

Unpack and launch Eclipse

After downloading and extracting Eclipse (zip or tar file), there's nothing else to the installation process from the Eclipse side. In the extracted folder, simply run the eclipse executable file. It will ask for a workspace location; the default is fine.

Install CDT

Skip this step if you installed the "Eclipse IDE for C/C++ Developers." Here you will install the CDT (C/C++ Development Toolkit). You can do this from the Eclipse UI as follows:

1. Help > Software Updates > Find and Install...
2. Select "Search for new features to install", and click "Next"
3. Select "New Remote site" and enter
   • Name: "CDT update site"
   • URL: enter "http://download.eclipse.org/tools/cdt/releases/europa"
   • If you plan to install PTP right now, you can install CDT & PTP together: enter its update site URL from step 3 'Install PTP' below
4. Make sure the "CDT update site" item is checked, and select Finish.
5. Select a mirror site if asked.
6. When the Update site is loaded you see the "Search Results" dialog.
7. Expand the "CDT update site" item, expand "CDT Main Features" and check at least the "C and C++ Development Tools" item.
   • Check any other optional items you like.
8. Click Next, accept the license terms, Click Next, and click Finish.
   • Eclipse proceeds to download the components you selected.
9. On the Verification page, click "Install All."
10. When it's done, click "Yes" to restart the Eclipse workbench.

Note: if you have trouble with remote update sites, you can download an archive of the update site on the CDT builds page.

Install PTP on local machine

Install PTP from the Eclipse UI as follows. This will install the PTP components into Eclipse on your local machine. If you don't intend to run parallel applications locally, there is no need to install any resource managers.

1. Help > Software Updates > Find and Install...
2. Select "Search for new features to install", and click "Next"
3. You can use the remote update site, or a local archived update site previously downloaded:
   1. To use the remote update site, Select "New Remote site" and enter
      • Name: "PTP Update site"
      • URL: enter "http://download.eclipse.org/tools/ptp/releases/2.0"
   2. To use the archived update site, Select "Archived Site" and enter the location of the archived update file you can download from the builds page
4. Make sure the "PTP Update site " item is checked, (you can uncheck CDT if you already installed it and retarted eclipse) and select Finish.
5. Select a mirror site if asked.
6. When the Update site is loaded you see the "Search Results" dialog.
7. Expand the "PTP Update site" item, expand "Parallel Tools Platform 2.0" and check the features you want:
   • Parallel Tools Platform End-User Runtime (required)
   • PTP Parallel Language Development Tools (optional, but recommended)
   • PTP Common External Components (required, platform-specific fragments, proxy & util libraries)
   • PTP Remote Tools Enabler (optional for remote, must have PTP Remote Tools installed)
   • PTP RSE Enabler (optional for remote, must have RSE installed )
   • PTP Support for Open MPI (optional, required to use Open MPI as Resource Manager)
   • PTP Support for IBM Parallel Environment (optional, required to use IBM PE as Resource Manager)
   • PTP Support for IBM LoadLeveler (optional, required to use IBM LL as Resource Manager)
   • PTP Remote Tools (optional)
8. Click Next, accept the license terms, Click Next, and click Finish.
   • Eclipse proceeds to download the components you selected.
9. On the Verification page, click "Install All."
10. When it's done, click "Yes" to restart the Eclipse workbench.

Note: if you have trouble with remote update sites, you can download an archive of the update site on the PTP Builds page.

Installation Examples

User wishes to use PTP with the parallel language development tools, and use Remote Tools for remote access. Parallel jobs will be submitted using OpenMPI. Select the following components:

• Parallel Tools Platform End-User Runtime
• PTP Parallel Language Development Tools
• PTP Common External Components
• PTP Remote Tools Enabler
• PTP Support for Open MPI
• PTP Remote Tools

User wishes to use PTP directly on a machine with LoadLeveler installed (no remote services required). PLDT is not required. Select the following components:

• Parallel Tools Platform End-User Runtime
• PTP Common External Components
• PTP Support for IBM LoadLeveler

Users wishes to use PTP with a remote machine that has IBM PE installed. PLDT is required. RSE has already been installed in Eclipse. Select the following components:

• Parallel Tools Platform End-User Runtime
• PTP Parallel Language Development Tools
• PTP Common External Components
• PTP RSE Enabler

User wishes to use Remote Tools only (without PTP). Select the following component:

• PTP Remote Tools

Remote Proxy Agent Installation

If you wish to control a resource manager running on a remote system, you will need to install the corresponding proxy agent on the remote machine.

1. Download the resource manager proxy agents onto the remote machine from http://eclipse.org/ptp/downloads.php
2. Untar (or unzip) the file.
3. Follow the instructions below for building the proxy agents.

Building Proxy Agents

Locate the plug-in org.eclipse.ptp.os.arch_x.y.z.###, where os and arch correspond to your operating system and architecture respectively, x.y.z corresponds to the release version of that particular plugin (e.g. 2.0.0), and ### corresponds to the build date (e.g. 200804010706). The plugin will either be in your Eclipse installation plugins directory if you installed the resource managers locally, or in the location you unzipped the proxy agents in the previous step.

For example,

 cd  <eclipse-install-location>/eclipse/plugins/org.eclipse.ptp.linux.x86_2.0.0.200804010706

After changing to this directory, run the build command:

   sh BUILD

This should build and install the executables. Check the bin directory to see if there are executables in it (e.g. ptp_orte_proxy and sdm). If you see any errors here, please refer to the Trouble Shooting section below. If you installed Open MPI in a non-standard location, see below also for an alternate method of building Open MPI.

Note: be sure that LD_LIBRARY_PATH is set (to point to OpenMPI libs, e.g. /usr/local/lib) for the environment in which eclipse is launched, and the environment in which the above build is performed. PATH should point to mpi as well, e.g. /usr/local/bin.

When you create a resource manager, you will need to navigate to the bin directory and select the appropriate resource manager proxy agent. When you configure a debug launch configuration, you will need to select the sdm executable in this directory.

Building PE and LoadLeveler Proxies

If you are installing the PE and/or LoadLeveler proxies on a remote AIX or Linux system, then you should use these steps to transfer the source code to the remote system and build the proxies on the remote system.

• Install PTP and prereqs (RSE, CDT) on your local system using Eclipse install procedures
  • Eclipse 3.3, CDT 4.0.3, RSE 2.0.3 for PTP 2.0
  • Eclipse 3.4, CDT 5.0.1, RSE 3.0 for PTP 2.1
    • CDT, RSE, PTP combined in PTP download for PTP 2.1
• 'cd' to the plugins subdirectory in your Eclipse installation directory on your local system.
• Create archive from plugins directory using latest date version of the following directories
  • org.eclipse.ptp.proxy.* (required)
  • org.eclipse.ptp.utils.* (required)
  • org.eclipse.ptp.rm.ibm.pe.proxy.* (for PE)
  • org.eclipse.ptp.rm.ibm.ll.proxy.* (for LoadLeveler)
  • org.eclipse.ptp.aix.ppc.* (AIX)
  • org.eclipse.ptp.linux.ppc.* (Linux)
• Upload archive to remote machine and unpack archive
• Make sure LoadLeveler is installed if needed
• cd to org.eclipse.ptp.{aix,linux}.ppc.*
  • Use directory matching OS
• Invoke BUILD
• Verify that proxies are built correctly
• PE and LoadLeveler proxies can be built by hand
• Build org.eclipse.ptp.utils and org.eclipse.ptp.proxy first
  • Build by invoking ./configure;make in each directory
  • ./configure –help for more options, including alternate LoadLeveler installation directory
• Proxies can be installed in common directory (/usr/local)

Test your installation

Start Eclipse. (You may need to use the -clean parameter to force it to recognize new features and plug-ins).

Create e.g. a small MPI C project. Open the PTP Runtime perspective (Window>Open Perspective>Other...). The runtime service will start and you should see a machine and one or more nodes appear in the 'Machines View'. Add a resource manager and start it. Create a launch configuration (Run -> Open Run Dialog...) and create a launch configuration for a Parallel Application.

More details on using PTP can be found in Help>Help Contents. Click on "PTP: Parallel Tools Platform ". The PTP help is also available online.

Known Issues

Resource Manager

1. There are synchronization issues when two (or more) remote resource managers attempt to start simultaneously. This only happens when PTP is first launched, and attempts to reconnect to resource managers that were running when Eclipse was closed.

Reproduce By: Start two remote resource managers that use remote tools connections. Exit from Eclipse. Restart Eclipse.

Workaround: Shutdown resource managers before exiting Eclipse

2. If two resource managers share the same remote connection, stopping one will also stop the other.

Reproduce By: Create two resource managers that use the same remote connection. Start both resource managers. Stop one resource manager.

Workaround: None. This is expected behavior since shutting down a resource manager also closes the connection.

Debugger

1. Accelerator keys (F5, F6) operate on the current process selected in Debug view. There is currently no way to use these keys for the Parallel Debug view.

Launch configurations

1. Certain sequences can result in a blank Resources page in the Resources tab.

Reproduce By: Start a resource manager. Open launch configuration and choose resource manager from Main tab. Select Resources tab and fill in some resources. Close launch configuration and shutdown resource manager. Start a different resource manager. Open the same launch configuration and switch to Resources tab, then switch back to main tab. Set the launch config to new the new resource manager, then switch to resources.

Workaround: None.

Troubleshooting

1. I get lots of error messages about missing include files when running the BUILD script.
   • This is because OpenMPI was not configured to include the development headers. You need to re-run the configure command for the OpenMPI installation specifying the --with-devel-headers option.
2. I get errors like "configure: error: utils directory not found" when running the BUILD script.
   • Make sure that you don't have old versions of PTP along side the version you're building. The BUILD script will get confused if there is more than one version of PTP. (i.e. remove any older versions from your plugins directory)
3. I installed OpenMPI in a non-standard location. Do I need to do anything differently?
   • Yes. The standard location is /usr/local. If you install it to, e.g., /usr/local/openmpi, then when you build the platform-specific PTP code, instead of sh BUILD you should use the following command: CFLAGS=-I/usr/local/openmpi/include LDFLAGS="-L/usr/local/openmpi/lib -Wl,-rpath,/usr/local/openmpi/lib" sh BUILD
4. When I create a new C project with the new project wizard, nothing happens. The dialog doesn't show.
   • Make sure you have at least CDT 4.0.2. Version 4.0.1 and previous builds will not work with the PLDT new project wizards.
5. Note: The application must be able to be run from the command line without supplying additional arguments (such as -x). In particular, the PATH and LD_LIBRARY_PATH must be set correctly on the frontend and all the nodes. This limitation will be removed in PTP 2.1.
   • Unfortunately the current (2.0) version of PTP does not allow you to alter the mpirun command line (in order to add '-x LD_LIBRARY_PATH', for example), so the only way to get around this problem is to add the path to the libpetsc.so library to the LD_LIBRARY_PATH variable in your .bashrc. The next version of PTP (2.1, due around August) will remove this restriction.
   • The debugger does not handle this failure condition very well. We'll be looking into improving the error messages in 2.1.
6. Note: The debugger has some issues when used on C++ code. We'll determine what these problems are and make sure they're fixed in 2.1.

See also the older 1.1.1 Release notes troubleshooting section

FAQ

1. Simulating multiple nodes with OpenMPI: If you are testing on a single node, edit <openmpi_install>/etc/openmpi-default-hostfile (default is /usr/local/etc/openmpi-default-hostfile) and add lines containing 'node0', 'node1', etc. Edit /etc/hosts and add a localhost entry for each name you added to the openmpi-default-hostfile. So, for 'node0', add '127.0.0.1 node0' to /etc/hosts. This will simulate a multi-node machine.
2. How do I debug my program remotely?
   • Currently you must move the executable to the remote target(s) yourself. The run/debug launch configuration contains an entry for the location, on the remote machine, of the executable to run/debug. The source in your local machine (running eclipse and PTP) must match the executable on the remote machine for source-level debugging to work correctly. This will change when CDT more completely handles remote resources, hopefully by CDT 5.0 (summer '08)
3. How do I launch eclipse?
   • Normally just run the eclipse executable - however, you may want to do this from a command line instead of from a "shortcut" - some OS's or windowing systems do not properly send the environment information. To make sure the PATH and LD_LIBRARY_PATH get correctly sent to the eclipse enviornment, you may want to open a shell and run "./eclipse" -- See also the troubleshooting section of the PTP 1.1.1 release notes for more information.