Parallel Tools Platform FAQ

Q: How do I download PTP?

PTP can be installed in two ways:

• from the all-in-one package "Eclipse IDE for Scientific Computing" on the Eclipse downloads page
• or installed into an existing eclipse via the Eclipse Installation Manager (Help > Install new software...).

See the PTP download page; check regularly there for new releases, too.

The current released version of PTP is shown on the Eclipse PTP web page at http://eclipse.org/ptp

See the PTP release notes for detailed instructions on how to install PTP.

Q: How do I find out more about Synchronized Projects and System Monitoring?

There are other FAQ pages with Frequently asked questions about

• PTP Synchronized Projects and
• PTP System Monitoring

Q: What version of MPI do I need to run PTP?

PTP supports the following MPI runtimes:

• Open MPI 1.2.x and higher. Note: debugging *does not work* for Open MPI version 2.x, 3.0.x and 3.1.0. Use a later version to debug.
• MPICH2 1.0.6p1
• IBM's Parallel Environment (PE)

The most recent OpenMPI (generally) is recommended for the most recent release of PTP, although it may work with older releases.

For the Parallel Language Development Tools only, there should be no difference between the different versions of MPI, such as OpenMPI and LamMPI. You just need to configure your project to be built using mpicc rather than gcc (for MPI releases that require that; most do), and to (optionally) specify the include path to the MPI header files. This should be described in the PLDT help.

Q: What platforms and runtimes are supported by PTP now?

The current release of PTP runs on Linux, MacOS X, and Windows. PTP only supports target systems running Linux, MacOS X, and some kinds of Unix (e.g. AIX).

The Parallel Language Development Tools will run anywhere Eclipse and CDT will run. It may require a version of MPI (specifically, its header files) to aid in the search for MPI artifacts. And an OpenMP header file, if OpenMP analysis is needed. This means you can develop and analyze source files but you can't launch them.

The Remote Development Tools can be used on Linux, MacOS X, and Windows.

Q: How do I install Open MPI?

You may already have it installed. Try running the following command and check the output:

   ompi_info

If this doesn't work, download the latest version from the Open MPI site.

Unpack the distribution and run the commands (note that you may have to add extra options to configure to suit your local site - see the Open MPI help documentation for more information):

   ./configure --prefix=/path/to/your/ompi/install
   make
   make install

At this point it's a good idea to run a small mpi program to test you have installed Open MPI correctly.

mpicc -o testMPI testMPI.c 
mpirun -np 2 testMPI

Q: How do I build PTP myself?

See the PTP Environment Setup instructions

Q: How do I ask questions about PTP?

Subscribe to the PTP Mailing lists, especially ptp-user, and ask questions there. This is an excellent place to ask new-user questions. The ptp-user list is watched by most of our PTP developers as well as many users, and you can share in their experience. See http://www.eclipse.org/ptp/mailing_lists.php

Q: How do find out more about PTP?

• Read the PTP Help documentation from the PTP Website or download it and read the help within your PTP installation.
• Download slides from previous PTP tutorials
• Ask questions on the mailing lists: See http://www.eclipse.org/ptp/mailing_lists.php

Q: What's the most recent version of PTP?

The current released version is 2019-03.

See the announcements on http://eclipse.org/ptp which should indicate the latest release number.

Q: How do I install the most recent released version of PTP?

There is an all-in-one installation package, and update sites for you to install PTP into your own installation of Eclipse.

The all-in-one installation is the easiest way to install PTP and it can be installed

• from the all-in-one package "Eclipse IDE for Scientific Computing" on the Eclipse downloads page

Use one of the update sites listed in Help > Install New Software.

Q: I want to live a bit on the bleeding edge. How do I install a more recent PTP build, e.g. to get a bug fix ?

Go to the PTP builds page and find the page for your release (e.g. if you have 6.0.x, find the page with the largest value of x) and download the archive file (ptp-master-xxx) then use that in the Help > Install New Software... dialog.

Q: When I launch a parallel job how do I stop it from building everything in my workspace ?

Open Preferences (Window > Preferences or on a Mac, Eclipse > Preferences), Open Run/Debug / Launching

Under "Run/Debug", select "Launching" and Uncheck "Build (if required) before launching"

Q: How do I create my own Target System Configuration for the scheduler on my remote system ?

See the next question.

Q: How do I customize an existing Target System Configuration?

A detailed guide to the XML Schema for the new Configurable Resource Manager, along with an introductory tutorial slide-set demonstrating some simple modifications to an existing XML definition, are now available at Target System Configuration.

• Go to File>Import..., then open Target System Configurations>Target Configuration From Plug-in.
• Choose the configuration you want to clone (e.g. PBS-Generic-Batch) from the combo.

Alternatively, if you already have an XML Definition file.

• Create a General project named "targetConfigurations" by doing: New > Project… > General > Project > Project Name: "targetConfigurations" > Finish
• Import the XML definition file by Selecting the "targetConfigurations" project > Import… > General > File System > Next > Browse… > Select Directory containing XML file > Check XML file > Finish

In either case this creates a new project in your workspace called "targetConfigurations" containing the XML definition.

• Open the targetConfigurations folder.
• Double click on the xml file to open it in an editor.
• Change the "name=" attribute in the <resource-manager-builder> element to be something unique.
• Make whatever other changes you require.
• Save the file.

To always use the latest configuration after you might have edited it:

• Go to Preferences then select Parallel Tools>Target Configurations
• Check the "Always load XML from URL" option.
• Open the run configuration dialog
• You should see the name you entered above in the list of configurations. If not, check that you picked a unique name.

More information:

• http://wiki.eclipse.org/PTP/workshops/September_2012 includes several presentations on related topics
• Adding Support for a new Resource Manager
• Monitoring system basics, and adding support for a new batch system
• PTP User Meeting of January 2014 covered creating a TSC and debugging

Q: How do I enable the Eclipse menu icons to show in Linux with GTK?

See this bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=293720

Q: How do I use PTP to connect to a remote system via an intermediate host (multi hop)

Use the "Advanced" section of the "New Connection" dialog to create an SSH proxy. This should be a connection to the intermediate host. Configure the main host as if you were connecting directly to the remote system.

Q: Why does the Project Explorer flag all directories to contain (git) changed or untracked files ?

The situation occurs if both, synchronization (via PTP) and revision control (via git) are used concurrently. Eventually, this is not an error, the Project Explorer is quite correct - there are untracked files. To make this effect disappear, open an editor for the file <project_root>/.gitignore and add the following lines.

 /.project
 /.cproject
 /.settings/
 .gitignore
 .ptp-sync-folder

Actually the last two lines change with the Eclipse version. '.gitignore' is needed for Eclipse Kepler. Instead of this filename, Eclipse Luna uses '.ptp-sync-folder'. Of course it doesn't hurt to add both. Then you are safe with either Eclipse version.

Q: Why should a general project be first converted to a C/C++ project and then to a synchronized project ?

The situation might occur if a git repository has been imported into Eclipse as a general project, but should now be converted to a synchronized C/C++ or Fortran project. There are two wizards available, one for conversion to a C/C++ project and one for conversion to a synchronized project. So these have to be called one after another, but the order matters.

The wizard for converting to a synchronized project shows an extra page for linking sync configurations to build configurations, if applied to a C/C++ or Fortran Project. If not dealt with on this wizard page, the linking of sync to build configurations would need to be added manually to the project settings. Thus the following recipe provides the smoothes path to a synchronized C/C++ or Fortran project.

• Convert to a C/C++ project.
• Convert to a Fortran project, if applicable
• Add an additional build configuration, if there will be different build configuratuions locally and remotely.
• Convert to a synchronized project, linking the sync configurations to the given build configurations.

Q: Eclipse complains "No such file or directory" when attempting a remote build for a managed project

Managed projects create their build directories. By default they are called "Debug" or "Release". These are created locally and then replicated to the remote host. Please check your replication filter settings and make sure that at least the makefile and make include files in these directories are being transferred to the remote side.

Q: Eclipse doesn't seem to find commands for [compiler, make, ...]

If you get an error, e.g. in the console, saying something cannot be found, such as:

 error
 /bin/sh: ifort: not found
 make: *** [test.o] Error 127

Then it likely means that the required command cannot be found in your PATH environment variable, e.g. ifort in this case. Note that the PATH environment variable the Eclipse uses can be different to the PATH set in your .profile or .cshrc.

If you are starting Eclipse from a shell make sure the path to the command is specified for the PATH environment variable in your login script. (Note: reference interactive shell vs. non-interactive shell FAQ item)

If you are starting Eclipse from a Linux window manager (Gnome, KDE, etc.) then consult your window manager documentation for information on how to set environment variables.

If you are starting Eclipse from the OS X window manager, the solution depends on the version of OS X.

For OS X 10.7 (Lion) and earlier:

PATH must be set by creating a file called environment.plist in a directory called .MacOSX in your home directory. The file should look like this, with [path_from_shell] replaced with the correct path, such as that obtained by the command "echo $PATH" in your shell.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>PATH</key>
  <string>[path_from_shell]</string>
  </dict>
</plist>

For OS X 10.8 (Mountain Lion):

The environment.plist method no longer works. Instead you have two choices:

1. Modify the environment for all applications globally. This is done by using the command 'launchctl setenv key value'. You will need to restart the application for the new environment to be picked up. Note that this only lasts until the next reboot. For a permanent fix, edit /etc/launchd.conf and add a 'setenv' line.
2. Modify the environment for the specific application. This involves editing the application Info.plist and adding an 'LSEnvironment' key. See LaunchServicesKeys for more information.

There are also tools such as EnvPane which may be helpful, and a good discussion here.

Q: I get the error "Failed to execute command: ompi_info -a --parseable" when I use the Open MPI target configuration on my Mac

This is because the "ompi_info" command is not in your PATH. See the previous FAQ item.

Q: When dealing with my remote system I see an error "Invalid java version or java not installed on "<connection name>""

See the other FAQ items about environment variables. Your userid isn't finding Java in your path.

Remote projects (not synchronized projects) need java on the remote system in order to start the indexing service. So make sure java is in your PATH, and it is accessible from a non-interactive shell (see FAQ item on this).

Q: Why is the Eclipse Editor is flagging bogus errors in my code ?

Flagging of code errors in the editor view is done by Codan (see http://help.eclipse.org/kepler/index.jsp?topic=%2Forg.eclipse.ptp.pldt.doc.user%2Fhtml%2Fcodan.html). In most cases, Codan is unable to work out the correct include paths or CPP macro settings. The general recipe to deal with the latter would be the following.

• If the MPI constants such as MPI_COMM_WORLD are erroneously flagged as error, use the next item in this list and add the directory of mpi.h (or mpif.h). This directory depends on the MPI you are using and the installation location of the MPI. The standard path for OpenMPI is /usr/include/openmpi.
• Check whether adding an include path to "project manager menu => Properties => C/C++ General => Paths and Symbols => Includes" solves your problem. You might want to grep for the flagged symbol in all files <*.h> in /usr/include or any sub directory and check whether the include path is already in the list. If you added a new include path, follow the instructions to re-index the project.
• Check whether defining a CPP variable in "project manager menu => Properties => C/C++ General => Paths and Symbols => Symbols" solves your problem. It might be that the include file is found but Codan gets entagled in some cascade of #ifdef-s. If you added a new symbol, follow the instructions to re-index the project.
• Disable Codan error flagging alltogether by eliminating the bug symbol in top row menu => Window => Preferrences => General => Editors => Text Editors => Annotations.
• Post a question in the CDT forum.

Q: How do I set remote include paths in a synchronized project?

For information on setting remote include paths explicitly to use those on the remote system using a UNC-style path (for eclipse indexer, e.g. using editor features), see Eclipse PTP Kepler tutorial, page "Editor-5" (PDF page 88 ) in PTP Kepler tutorial from SEA 2014

• Summary: set C/C++ General > Preprocessor Include Paths to "//<connection-name>/path/to/includes" (and set "File System Path" and check "Contains system headers" (Note: there is still a bug with this working correctly on Windows. See Bug 416658)

Q: How can I get the C/C++ indexer to use my remote include paths in a synchronized project?

Even if you have your project set to use a remote build configuration that contains the paths to the remote include files, the C/C++ indexer may find include files on the local (client) machine instead of the remote machine.

The problem here is the settings for the indexer itself. To fix this, you must set the indexer to use the active build configuration, instead of the default, which is usually the local build config. A further complication is that because of a bug in the indexer project settings, any change you make to it will not be preserved when you restart Eclipse (see Bug 205299). Fortunately, there is a simple work around: use the global settings for the C/C++ indexer instead of the project-specific settings; the global setting is preserved across restarts of Eclipse. To change this setting, use Window->Preferences->C/C++->Indexer then check "Use active build configuration".

Q: My remote or synchronized project doesn't find the remote environment variables correctly (Interactive vs. non-interactive shell)

Note that communication with your remote host is done via ssh. Even if you are connecting back to your local machine to run/debug locally, make sure you do this.

Remote Tools needs the proper environment variables in order to find java, compiler, make, etc.

Your environment may be set up correctly for an interactive shell but not for a non-interactive shell. You probably want to set up the same environment for an interactive and non-interactive shell. For example, for bash, there are at least two files ~/.profile and ~/.bashrc. Make sure you know which ones your system uses. Naturally this depends upon what shell you are running.

• ~/.profile or .bash_profile– read by ssh in interactive shell on login
• ~/.bashrc – read by ssh in non-interactive shell on login

Make sure the non-interactive shell is set up properly. Perhaps helpful reference(s):

• Setting SSH Remote Host Environment variables
• Try this from the command line of the machine eclipse is on:
  • ssh userid@remotehost command (e.g. ssh beth@myRemoteSystem.com ls)
  • try the command you are trying to do that is giving you problems, instead of 'ls'

An alternative (works for ksh for at least some of us) is to do the following:

• Modify sshd_config to have "PermitUserEnvironment yes" and reload sshd to use it. You'll need root access to do this.
• Added ~/.ssh/environment to your userid to contain the java path, for example, PATH=/usr/local/bin:/bin:/usr/bin:/opt/java/jre/bin

More notes from ptp-dev discussions

• It depends on which shell you use on the remote machine. Usually the default for Linux is set to bash, but some people like ksh, tcsh, etc. and have their default shell set to that.
  • tcsh uses ~/.tcshrc, and if that doesn't exist, it tries ~/.cshrc
  • ksh ... appears to use the same file as bash (and sh) - .profile
  • bash is recommended!
• Most of the above depends on system configuration for non-interactive shells. Most have it configured to read ~/.bashrc, but not all, and without this setup you may not be able to set the environment.
• Pure Remote Projects (not synchronized projects) need java on the remote server. Here's how to test:
  • from your workstation, tunnel via ssh to the remote machine:

ssh userid@myRemoteSystem.com which java 
ssh userid@myRemoteSystem.com java -version

Useful references

• http://www.understudy.net/custom.html - several different shells described, including interactive vs. login shells

Q: I get an 'Internal Error' when debugging an application running on Mac OS X

On Mac OS X Lion (and later) you may see an error similar to the following when you try to start the debugger locally:

Error on tasks: 0 - Internal Error
Reason: Debugger error: Unable to find Mach task port for process-id 51470: (os/kern) failure (0x5).

This is because Lion has introduced new security requirements for gdb to work. Note that gdb is installed by default as /usr/libexec/gdb/gdb-i386-apple-darwin. The /usr/bin/gdb command is just a script that invokes this executable.

Lion

Execute the following commands (this assumes that your user is in the procmod group, check using the 'id' command):

chgrp procmod /usr/libexec/gdb/gdb-i386-apple-darwin
chmod g+s /usr/libexec/gdb/gdb-i386-apple-darwin

Mountain Lion

Running the following command appears to work:

sudo DevToolsSecurity -enable

Q: I get the message 'Unable to determine gdb version' when trying to debug a program

Mac OS X Mavericks uses lldb instead of gdb. In order to debug an application running on Mavericks, you first need to install gdb following these instructions. Only follow the Getting it to Work with Eclipse section if you wish to use the CDT debugger.

If you are not using Mavericks, make sure you have the "Path to backend debugger" preference set to the path to the gdb program, not the sdm. See this bug for more details.

Q: How do I debug the server part of PTP's system monitoring capability?

If the Active Jobs view is empty when you know jobs are running on the system, perhaps the commands queried from the monitoring system are not successful.

1. On the remote machine, go to the ".eclipsesettings" directory, located in your home directory
2. Create a file called ".LML_da_options" containing a single line "keeptmp=1" (no quotes).
3. Restart the monitor.
4. You should now find a directory called "tmp_<hostname>_<pid>" in the ".eclipsesettings" directory. It should contain an error log file, plus a bunch of other files. Check these files to see if you can see the cause of the error.
5. Remember to remove the ".LML_da_options" file once you have finished.

See also PTP System Monitoring FAQ

Q: What do I do when i get "Failed to Create the Java Virtual Machine" launching on win32?

The JVM cannot allocate enough memory with current eclipse settings. FInd your eclipse.ini file (in the same directory as the eclipse executable), and in the line e.g. -Xmx2048m ==> change it to a smaller number e.g. to -Xmx512m or -Xmx768m or -Xmx1024m

It could be that your eclipse will run out of memory on a machine with a relatively small amount of RAM. (Need to publish suggested memory minimums.)

Note that a 32bit windows app only can have 2GB memory total. Changing the -Xms and -Xmx values to:

-Xms512m
-Xmx1536m

may fix it (assuming your machine tolerates *that* much). Older machines, or with less memory, may need to go down to

-Xms40m
-Xmx512m

But you risk OutOfMemoryException later

Q: What systems does the PTP Parallel Debugger support?

For the parallel debugger to work, PTP must support the remote system for debug in the Target System Configuration / TSC. See the Release Notes for each release of PTP.

The PTP parallel debugger is implemented for:

• Generic TSCs:
  • Generic OpenMPI Interactive
  • Generic MPICH2 Interactive
  • Generic Remote Interactive
  • IBM Parallel Environment
  • IBM Platform LSF Interactive
  • IBM Platform MPI
• Specific system TSCs:
  • Trestles Torque OpenMPI Interactive

How to tell? In the Target System Configuration xml file, "submit-interactive-debug" will be found if the PTP parallel debugger is supported. (in one of the org.eclipse.ptp.rm.* plugins)

Q: The PTP Debugger hangs with "Operation in progress..." using Open MPI

You might have an unsupported version of Open MPI. In particular, version 2.x and 3.0.x do not work due to a bug in mpirun. You will have to update to a different version of Open MPI to use the debugger. See this issue for more details.