Using Buckminster with Non Java Projects

< To: Buckminster Project

Purpose

While trying to use Buckminster for both my Java and C++ projects, I realized that Java projects within Eclipse were pretty straightforward but C/C++ projects were much more difficult to manage. As a result some new actors were proposed and then submitted to the Buckminster project. This example shows how to use these.

Why is it so hard to use Buckminster with non Java projects ?

Component materialization is much harder in non Java environments because :

• there are no standard way to describe C/C++ projects so you have to write the CSpec file anyway ( contrary to Eclipse Java projects which provides a way to auto generate the CSpec )
• compiling, testing, your code requires to execute programs from outside (shell scripts, compiler, unit tests and so forth).
• some of the resources you need to materialize are libraries that are only available as zipped url resources.
  • which implies you have to download and unzip them : Buckminster cannot do that for the moment despites its exactly what you expect it to do. So to overcome this issue you have to write ant tasks.

Work flow currently used

The toolchain I created to manage C/C++ projects dependencies is as follows :

• Create Spec components to describe dependencies
• Specify actions like build / clean / rebuild
  • those actions have prerequisites pointing to other components (eg: the path to the libTiff include folder)
• Write ant scripts called from buckminster to
  • retrieve libraries from urls or retrieve files from archives pointed by url
  • execute commands with specific environment variables (eg. calling the compiler with paths to libraries)
    • those environments variables are retrieved from Buckminster's action's prerequisites section via Ant actor API (Buckminster)
• Create the script to compile the code ( Makefile or Boost Jamfile or SCons file )

Actors

The actors described below can be used with all types of components, but specifically created to provide a better experience to those that work with non Java components.

Executor Actor

This actor adds the possibility to execute programs directly from the CSpec without writing Ant Scripts.

Features

• Executes programs or shell scripts
• Sets arguments
  • arguments can refer to prerequisites
• Sets environment variables for the execution
  • environments variables values can refer to prerequisites
• Sets execution directory

Example

<cs:public name="build" actor="executor">
      <cs:actorProperties>
            <cs:property key="newenvironment" value="true"/>
            <cs:property key="env" value="TIFF_LIB=${tiff.lib};TIFF_INCLUDE=${tiff.include};
                                          VERSION=${VERSION};OSGI_VERSION=${buckminster.version}"/>
            <cs:property key="shell" value="${compiler} -j2 debug"/>
      </cs:actorProperties>
      <cs:properties>
            <cs:property key="VERSION" value="1.0.0"/>
      </cs:properties>
      <cs:prerequisites>
            <cs:attribute component="mingw-5.1.4" name="compiler" alias="compiler" />
            <cs:attribute component="libtiff-3.6.1" name="include" alias="tiff.include" />
            <cs:attribute component="libtiff-3.6.1" name="lib" alias="tiff.lib" />
      </cs:prerequisites>
</cs:public>

Valid properties

This syntax integrates into Buckminster's CSpec Actions.

• env : a list of semi-colon separated environment variable
• newenvironment : if set to true, the execution will start from a blank environment variables.
  • Note in early design development the opposite paradigm was chosen and one has to use inheritEnvVar to start from the system environment variables, now it's the default behavior.
• failonerror : if unset or set to true, the action will fail if execution returns non 0 return value. Default is true.
• exec : the program to execute
• execDir : the program execution directory (if not set : the component directory)
• shell : a batch file to execute

Note that you have to choose between exec or shell keyword.

All those properties can contain variables.

Those variables are set by surrounding properties or named path with ${}. See examples above.

Fetcher Actor

This actor is used to materialize resources into the component.

Features

• fetch single (for the moment)
• log to https sites
• unzip, untar, untargz, untarbz2 files
  • possibility to specify destination directory
  • possibility to filter the archive content
  • keep hierarchy or flatten folders

Example

<cs:public name="download" actor="fetcher">
   <cs:actorProperties>
      <cs:property key="dir" value="data"/>
      <cs:property key="options" value="uncompress;flatten;include=*.html;include=*.c"/>
      <cs:property key="url" value="http://download.osgeo.org/libtiff/tiff-3.8.2.tar.gz"/>
   </cs:actorProperties>
</cs:public>

Valid Properties

• url : the resource to fetch
• dir : the directory where the file(s) should be placed
  • if not set dir points to component's location

If the resource you are accessing is protected with basic authentication, you can also specify :

• login : the login to use with basic authentication
• pass : the password to use with basic authentication

Note that both properties have to be set to enable authentication.

• options : a set of options to customize the fetch operation, valid options are :
  • uncompress : specify that the resource pointed by the url is an archive and that it should be uncompressed
  • flatten : valid only with uncompress. If specified the folder hierarchy is flatten
  • include : a regular expression to include specific files only. Note that you can specify multiple include directives in the options line. Regular expressions beginning with the '-' (minus sign) will be considered as an exclusion. See examples below :

Options samples

full options sample, uncompress the archive and flattens the folder hierarchy, including only files in the binaries directory that are not html files

uncompress;flatten;include=binaries/*;include=-*.html

flatten and include keywords will NOT work until the uncompress keyword is set, the following options line will simply copy the url resource locally.

flatten;include=binaries/*;include=-*.html