Why Buckminster ?

< To: Buckminster Project

Welcome to Buckminster

This document explains the motivation for the Buckminster framework and provides an introduction to the basic Buckminster concepts. The first section (How can Buckminster help me ?) provides a short introduction. If you like what see than you should move on to the following sections to explore some of the concepts in more detail.

How can Buckminster help me ?

Buckminster can help you with:

• Getting the things you need:
  • projects in source code form
  • binaries to your workspace, target platform or IDE
  • target platform
  • the tools you need in your IDE
  • the builders you need in a headless build
  • etc.
• Running actions on what you got
  • building parts, or everything to finished update site

Lets look into this in some more detail...

Solving a common problem

In its simplest form, Buckminster solves problems like the following:

• Your colleague says "OK, project Overland is ready for you to work on - just check out com.megacorp.overland from CVS and get started".
• You checkout com.megacorp.overland and discover that it has unresolved project dependencies.
• "Oh yeah," he says, "you have to checkout project overwater as well." Of course that doesn't completely solve the problem ...
• "Also you have to get the latest undersea.jar from the central FTP server."
• "But hey, it doesn't build ok!"
• "Oh, I forgot, you have to import this jar here, and rename it ... and let me see what else ..."
• Eventually, after a few rounds of this, you have the whole set of resources and you are ready to go to work.
• And then the story repeats itself as you want to be able to build everything in 'headless' fashion

In its simplest form, Buckminster is a tool for materializing a workspace.

In its broadest sense Buckminster puts the world of components at your fingertips by formalising component descriptions and dependencies, and materializing them into a context of your choice - one of which is a workspace.

Query the "cloud"

(Bear with us on these pictures - right now they are trivial, but soon Buckminster's complexity will make the pictures useful.)

As a component developer, your workspace is the set of components you are working on plus all of their prerequisite components. Under Buckminster, components can be many things, but one of the most important component types is Eclipse projects. In pictures, the workspace (the cloud) is initially empty, then you ask Buckminster to materialize the component (the box), after which the workspace contains the component.

What the simple example above boils down to is what Buckminster calls a component query (or CQUERY). It is the top-level entry point to the Buckminster framework. You are saying "I want "that" component". Buckminster provides everything necessary to ask that question (or express that query) AND materialize "that" component to a location of your choice. Actually, what you are requesting and what Buckminster allows you to express is more like:

• "I want the latest version of "that" component" OR
• "I want version x.y.z or a later versions of "that" component" OR
• "I want version x.y.z or a later versions of "that" component in source form" OR
• ... and so on

Things cannot possibly be that simple ?

Component A is the one you are working on; components B, C & D are required to be in the workspace.

This is indeed not a trivial problem. We can continue with the example queries above and extend them with one more important condition. You really are asking "I want the latest version of "that" component AND I don't want to worry about all dependencies".

Using the simple diagram above, let's assume that component A has dependencies on B & C which in turn have dependencies themselves: then the actual result is something like the example shown in the diagram.

Let's add a bit more context to this simple picture and let's assume your request for "that" component A should result in a project in your workspace which also requires B, C & D to be built. This is the point where Buckminster really shines.

In order to do so, Buckminster will have to understand a lot of things about components. We say "understand" because Buckminster interprets existing component information to perform its work. This information comes in many different forms such as plugin.xml or feature.xml files, POM files in Maven, etc. Buckminster understands them all, and translates them into a Buckminster component specification (or CSPEC). It is also able to learn about new ones.

The component infomation captured in those files (i.e. a plugin.xml or POM) will eventually allow Buckminster to discover that component A also needs B, C & D. Buckminster will have to resolve all of those dependencies before it knows exactly what it has to materialize into your workspace. Remember, that you just asked for component A but, after the Buckminster resolver has done its work, it will be able to tell you that you also need B, C & D.

But how does Buckminster know about all the components in the "cloud" ?

The Buckminster resolver is a bit like placing an order to an automated shopping system. You tell Buckminster what you want, and where you want Buckminster to put it when Buckminster finds it (I want component A as a project, and put it into my workspace without build errors). Like an automated shopping system, Buckminster needs a navigation map of possible shops and suppliers to help it do this job. Buckminster calls this map a resource map (or RMAP).

The resource map contains a component index and location descriptions (repository types and addresses). Buckminster looks up the component index which points to a location that Buckminster can visit to find components and their associated component information. As on any map, there are usually alternative routes. The resource map captures this as well. You may prefer Buckminster to use quickest route (get a binary version of D from a remote repository) or the shortest (get a source version of D from your local repository and build a binary).

Buckminster matches the information in the resource map against your request for "that component A with all all dependencies resolved and oh, I prefer to use stuff from a local repository". The component query and resource map together with Buckminster's understanding of components is sufficient to produce a "component shopping list".

Buckminster's "component shopping list"

Buckminster calls this a bill of materials (or BOM). With regard to our analogy of Buckminster as an automated shopping system, it may be that in placing your order for item A, it turns out that you actually are going to need items B, C and D as well (otherwise item A won't work at all). The BOM would be the computed full list of items and the route and location to get each one of them. It is a set of things to get and instructions on how to to get them. The Buckminster materializer understands this, and performs the final step: it actually physically gets the things you want and those you also need as a consequence.

For example, component B may require a binary version of D and, as part of the materialization process, Buckminster may first be required to obtain a source version of D and then build it. All of this is captured in the BOM. The BOM itself can also stand alone (as a saved file): to start a Buckminster materialization, you can either issue a component query or give it a previously produced bill of materials. In the context of your daily work this means that you can either share a CQUERY (and associated RMAP), or a (previously computed) BOM with your colleagues in your team, if they want to get "that" component A as well.

As a final twist, Buckminster also recognizes that the content of the same bill of materials should be materialized into different locations depending on your work context, system setup or personal preferences (some of your team members may run on Linux, others on Windows). The materializer obtains this information from a materialization specification (or MSPEC) which is associated with a bill of materials. With regard to our analogy of a shopping list, we can say that two shopping baskets may contain the same items but the items will be put on different shelves when you get them to their respective homes.

To read some more details about BOMs and MSPECs, skip to here.

Is Buckminster just another build system?

Buckminster extends other build systems and associated repositories, like ant and maven. It does not replace them: any build scripts and processes which you already have in place do not need to be replaced when using Buckminster. Rather, Buckminister allows you to use and fetch collections of software components from various repositories (whether they be for example maven, cvs, subversion or just simple file structures), bringing them into your workspace (or somewhere else, if thats what you want Buckminster to do for you). Equally, Buckminster alllows you to publish (the Buckminster specifications for) interesting sets of components which you feel other people may also find interesting and want to use.

What is Buckminster actually useful for ?

Well, we hope that you understand the general build and materialization problem we just discussed in this section. As software componentization becomes increasing common in the industry, the problem of assembling a full framework, or a full application, which involves contributions from several different (e.g. open source) projects, becomes more complex and error prone. Alternatively you may work for a software vendor, and have to assemble your product, or products, from multiple internal (e.g. closed, and possibly also external open source) projects within your company.

Using Buckminster, you can publish component configurations, and be confident that your users - or consumers - will obtain the configuration which you specified for them. You can, if you wish, be highly specific and ensure that every consumer gets precisely the exact configuration you specify. The basic instructions to Buckminster are CQUERY and RMAP, as discussed above; but Bills Of Materials (BOMs), which are automatically generated by Buckminster, enable precise prescriptions to the materialization mechanism.

Components can be binary code and/or libraries. But they can also be source code, documentation, test suites: a Buckminster component is just a container for software information.

The materialization mechanism of Buckminster provides an installation facility: you can publish a configuration, and for each of your users (consumers) Buckminster will not only find the correct assembly of components, but also install them as necessary onto your users' machines, using appropriate policies to replace or re-use previously installed components. If you wish to be more sophisticated, automated installation to remote machines is also possible. Buckminster Materialization Specifications (MSPECs) allow predefined control, and specification of various policies, to control the installation procedures.

Buckminster also enables virtual distributions (virtual distros). Here, a virtual distro means a software distribution published on a server, typically consisting of many inter-dependent software components, which is published to an audience or community of users; and then can be materialized (downloaded) by users (consumers) and (remotely, if required) installed. The distro is virtual in the sense that the publishing server does NOT store any of the components itself: instead it knows where to find each component, across a variety of storage sites and repositories, whether internal to your company or site, or worldwide. Sometimes "virtual distro" however is taken as having a different meaning: a package of operating system software hosted above a native operating system. Such "virtual hosting" is quite different from the "virtual distribution" capabilities enabled by Buckminster.

The ability to devise and publish interesting new configurations of components, and then to easily enable users to materialize these (in practice, a single mouse click materialization is possible), will IMHO accelerate the componentization of the software industry.

Buckminster: more of the story

Once you've grasped that Buckminster materializes components, you will be asking yourself "What are components ?", "Where does Buckminster get the components ?" and "How do my components fit into this ?". We will address those and other questions in more detail in the following sections.

Actually, what do you mean by component ?

We have been talking about components in our introduction, assuming that every reader will be able to relate a common notion to this term. Buckminter's view of components is very broad. Basically, in the Buckminster parlance, a component represents a collection of files, or sometimes just one single file. For a large number of Buckminster users, a component will be an Eclipse project, but components are a more extensive concept than that - components can be external to Eclipse, components can have attributes, dependencies, perform actions, etc.

Buckminster provides its own language to describes components: it expresses all knowledge about a component in a Component Specification (CSPEC), which in the most general case is stored in an XML file following the Component Specification XML Schema, inside the component.

You will probably ask yourself now how much effort it will be to provide a CSPEC for a component such as an Eclipse project. The answer is "Not much at all" because Buckminster will do the job for you. As mentioned earlier, for many component types the required information is already available through other means (plugin.xml, feature.xml, POM files, etc): in these cases, Buckminster will translate such information on-the-fly and there therefore is no need to repeat what is already known. The CSPEC is a bit like the Esperanto of component specification languages, and Buckminster is the universal translator that understands all of them. It can also learn about new ones by means of its extension points.

The component specification captures everything that Buckminster needs to know in order to resolve and materilaize components. One of the most important pieces of information in this context is dependencies.

Components Do Not Stand Alone: Dependencies, Prerequisites and Actions

File:Dependencies of A.gif

A component has zero or more dependencies

It is rarely the case that a component, such as an Eclipse project you are working on, is self-contained. Components may depend on other components: however a fundamental concept in Buckminster is that it is not until you actually do something with a component, that the prerequisites arise. Depending on what you do, you might need different things.

As an example, for a compilation-action of component A you may need interfaces (or header files of some sort) from component B - but you do not need the rest of B; and later, when running the system, you may need B in compiled binary loadable library form - but you are not interested in headers or the source of B. Buckminster separates the two concepts - the term dependency denotes the overall dependencies (every dependency), and the term prerequisite is used to denote that there is some action that has a dependency (what is needed for a particular action).

Dependency graph for component A

So, as an example; when using Buckminster lingo, you say: "A has a dependency on B", what you really are saying is: "There are actions in A that require B".

Each component can specify a variety of attribute types. Such an attribute can either access data directly, or trigger an action which yields the data. Thus, you can instruct Buckminster that "I only need this from that component". Attributes can also be grouped together to help simplify common activities.

As you would expect, Buckminster finds the transitive closure of the dependency graph when it materializes a workspace. The closure will always be in the form of a Directed Acyclic Graph (DAG). In our pictorial example, component A requires components B and C, and component B requires component D, and component C requires component D, so Buckminster will load all four into the workspace.

As you may have figured out, this is a simplification of what is really going on. Actually, some action was invoked on A, triggering the prerequisites for that action - something is needed from B, and C, and when obtaining those things from B, and C, those actions in turn have prerequisites on something obtained from D.

A component's dependencies, actions, and the actions' prerequisites are all expressed in the Component Specification (CSPEC), and serve as a script and/or set of constraints that Buckminster observes when resolving and materializing components. If you are interested reading more about dependencies, prerequisites, actions or attributes, then skip to further below.

Buckminster has extensive, and configurable, support for versioning of components: some further details are here.

Tell me more about how Buckminster does the CSPEC creation for me

When a component already has meta data (e.g. Maven or Eclipse PDE), this metadata is read and translated into a CSPEC that resides in the project model (i.e. in memory). It is possible to write this generated CSPEC to an XML file for viewing or other purposes. By generating the CSPEC on the fly, there is no need to maintain the same meta data in more than one place - the natural place is in the original component where the data is required anyway.

Buckminster needs two things to do this: knowledge about the component type and its associated meta-data; and a component reader with knowledge about the correct protocol to obtain this component meta-data, depending on in which repository (and possible revision control system) in which the component resides.

ComponentReader and ComponentType cooperate to read component meta data and translates it into an in-memory CSPEC used by Buckminster. The in-memory CSPEC can also be viewed or exported to a CSPEC file in XML format.

The Buckminster CSPEC, is a language capable of capturing a majority of the component languages in use. For a list of which comoponent languages are currently supported, skip to here. Sometimes the metadata available to Buckminster is not completely sufficient, in which case a Component Specification Extension (CSPEX) must be manually written: skip to here if you are interested.

Mapping the "cloud": where did you say my components were ?

While we have familiarized ourselves with Buckminster's understanding of components, and it's ability to handle component of different types, it has already been mentioned that Buckminster not only needs to know what components are, but also where they are.

There are two things to be said about where components are: one is the physical location or address of the component; and the other is the context of that location. Components are stored in some type of repository, and Buckminster needs to know about the type of repository in order to get the component and its meta-data. Both pieces of information are captured in the Buckminster resource map (RMAP).

How Does Buckminster Find Component Repositories ?

Component A being retrieved from a CVS repository and the required component B from an FTP server.

Components can be retrieved from a variety of different repositories. The most common repositories are repositories such as CVS, SVN or Maven, but a local filesystem or an FTP server are valid repositories as well. In fact, almost anything (databases, file systems, auto-generator programs, etc) can be used by Buckminster as long as it has a corresponding component reader. We have already mentioned the currently supported reader types and referred to Buckminster's org.eclipse.buckminster.core.readerTypes extension point.

Of course the story is a bit more complicated than the simple picture. So the first question you're asking yourself probably is how does Buckminster know which component is stored in which repository ?

The RMAP maps components to storage locations.

There are actually two concepts: the component and its storage location. Everything we know about a component, including its name, but regardless of its actual location, is stored in a CSPEC. When Buckminster tries to resolve and materialize components it needs to map the component name to a location. Separating these two concepts is like separating interface and implementation in good software design. Buckminster does this separation through a level of indirection known as a resource map (RMAP).

The RMAP provides location information for families of components. When Buckminster needs to load component A, it looks in the RMAP to determine the repository containing A, then goes to that repository and loads A. Then because A requires B and C, Buckminster looks in the RMAP for information on B and C, goes to those repositories, loads those components, and so on.

Buckminster's approach to defining a location in an RMAP is a very flexible mechanism. It employs a component name matching mechanism that allows to define storage locations for families of components rather than single components. Thus one can say "find all the Apache components on the Apache ftp server" without having to specify "find Apache Struts on the Apache ftp server and find Apache Cocoon on the Apache ftp server and find ..." ad nausea.

But the same component can be in many different locations ?

Mapping one component to alternative locations.

This is another advantage that the separation of component and storage location delivers. It allows Buckminster to retrieve the same component from potentially many different storage locations. The resource map actually defines a search path of storage locations to search for a component. Thus, one can say "first look in my local repository, then look in my team's repository, then look in my corporate group repository, then look in the Eclipse public CVS repository".

In Buckminster parlance we will call these alternative locations providers of a component. The most important piece of information that a provider must specify is its address (a URL) and access details (such as a login for a CVS repository). But each provider may have to say a bit more about the properties of the component such as whether it provides it in source or binary form or whether it is mutable or not.

So in summary, the RMAP will map a component name to a search path which really is a list of locations. Each of those locations is described as a provider which gives us all the details required to get the component meta-data and the component itself. The list of providers represents a set of alternatives routes through the Buckminster resolution and materialization process and Buckminster will pick the most suitable provider depending on the preferences set. We will have to say a bit more about this when talking about component queries.

When are alternative locations or providers useful ?

Location dependent RMAPs with different mapping preferences.

One useful thing is for example to have different search paths for different distributed development sites. Thus, the RMAP for Team Stockholm would list the same component families as the RMAP for Team Winnipeg, but with different local repository servers. TCP round-trips from Canada to Sweden are slower than TCP round-trips from Canada to Canada, thus the Winnipeg team maintains a replicated CVS repository for certain components.

Buckminster controls through properties specified in the RMAP which allows both teams to use the same RMAP - the exact same file - but to define a different set of search paths depending on where it is used from. Obviously the central servers (probably the last entry in each search path) would point to the same central Swedish servers so that if the components were not cached locally, they would be fetched across the Atlantic.

Just ask for it: I want that component !

The sections describing the main components are getting shorter as we are approaching the end of this Buckminster introduction. This actually reflects the fact that once the major pieces illustrated above are in place you can lean back and just let Buckminster do its work. You just have to ask for the component that you want.

How do I query for a component ?

A Buckminster Component Query is the topmost input to the Buckminster materialization process. It specifies "the thing you want materialized" and provides guidelines for the resolve and materialization stages of the process. The component query is captured in a CQUERY.

The Buckminster IDE includes an appropriate editor, but a CQUERY can just as easily be created using your favourite XML editor. In its simplest instantiation a CQUERY refers to an RMAP that should be used to resolve the query; and specifies a so-called root request, which states the name and version (or range of versions) of the component that you want to resolve and materialize.

A CQUERY offers almost boundless opportunities for customizing and controlling the resolution and materialization process through what is called advisor nodes in a Buckminster CQUERY. You are able to state whether you want to skip certain components whether you want to trust local copies (perhaps materialized previously); whether you want to override certain settings, etc. The way those constraints are applied is similar to the way component names are mapped to locations: that is, through a component name pattern matching approach. This offers the same flexibility: you are able to define constraints for whole families of components rather than individual components.

Sharing a CQUERY and RMAP

Once you are done editing the query, you can save it and reuse it. It is also ideal to share with other members of a team (together with the RMAP). They can then easily re-create the same setup as you have created, which brings us back to the example we started with.

Once you have created a suitable CQUERY and RMAP, you can make them available to your team (or the public) by making the files available on a web site (actually, via URLs). Make sure the CQUERY contains the RMAP URL. Tell people to start Eclipse, use the Open File ... dialogue and simply type (copy/paste) the CQUERY URL into the Filename field, and then click Open. That will open the CQUERY editor, from which the configuration can be materialized.

Eclipse materializes from a published CQUERY and RMAP.

What should you do next? And more details if you want them now..

First, you should update your Eclipse with the latest version Buckminster. The installation guide is here.

Then consult the Buckminster User Guide to see some common example usage scenarios for Buckminster.

If you are just interested in a high level overview of some of the further details of Buckminster, then take a look at some of the sections below:

• Dependencies, Prerequisites, Attributes and Actions
• Versioning
• What component languages are supported?
• Extended Component Specifications (CSPEXs)
• Bills of Materials (BOMs) and Materialisation Specifications (MSPECs)
• Abstract Components

Dependencies, Prerequisites, Attributes and Actions

Dependencies and Prerequisites - why the difference?

You may wonder why dependencies are declared separately from prerequisites of actions. Surely, the dependencies could be figured out by just following the prerequisites ?

The reason for separating the two is because at the component level, many prerequisites will refer to the same dependency and there is often a lot to say about a dependency that applies regardless of the actions associated with it - such as which version of a particular component is wanted. Specifying this multiple times, once per prerequisite, would result in a duplication of dependency information across a component specification: it would potentially be error-prone, as it would be easy to make a mistake and accidently specify different versions of the same component for alternative actions.

In summary, a dependency is specified at the component level and can be referenced by many prerequisites. A prerequisite will specify in detail why an action has a particular component dependency and what is actually required from that component in order to satisfy the prerequisite.

Prerequisites and Attributes

Remember we said that prerequisites are declared when a component needs something from another component to perform an action. Let's start with an example:

A compile action in A has a prerequisite on headers in B.

We know that in order to compile component A, we need access to the header files/interfaces provided by component B. To declare this we add a compile action to component A's CSPEC, and add a prerequisite for component B's "headers" to this action. In component B we need to declare the corresponding "headers". How is this done?

A component can specify a variety of attribute types. Such an attribute can either be data directly, or trigger an action producing the data. An attribute has a name that is unique within a component. Prerequisites simply refer to an attribute in a component by its unique name.

How are attributes specified ?

A component attribute in Buckminster sort of works as a getter method (using Java/OO terminology), and there are three approaches in a Buckminster component specification to declare attributes. They vary with regard to granularity and the means of obtaining the attribute:

• artifact - this is best described as an atomic attribute definition; it specifies none, one, or several paths to either individual files or directories (denoted by a path ending with a slash (/)).
• action - this is an attribute tied to an action that needs to be triggered when this attribute is requested (either by a component or a group or another action); it can produce a path structure on-the-fly returned as the value of the attribute.
• group - this is a group of artifacts, although any type of attribute (artifact, groups, actions) could be grouped to describe an attribute.

So, in our example, component B has an attribute called "headers" which is an artifact element with the path to the directory with headers.

Private and Public Attributes

When an attribute has a prerequisite on another attribute within the same component, it is said to be local. When a prerequisite needs to reference an attribute in another component, it is said to be an external reference. The external reference uses the component name, and attribute name.

The "headers" attribute has a local private prerequisite that headers are generated using an action.

Components can declare attributes to be private or public. The private attributes can only be used as prerequisistes for attributes in the same component. This makes it possible to create common datasets (artifacts and groups), or actions that are shared internally by the public attributes in the component. You do not have to repeat identical sequences and thus save time and space, creating a more robust, easier maintainable specification.

The most valuable use for private attributes is probably to hide internal complexities from users of the components. The user does not have to understand how certain sets of files are produced. For example that the header files may be generated from a UML model, but the user should not need to depend on that being the case.

Components as Attributes

Sometimes, an entire component is created as the result of some action in another component. Remember that Buckminster distinguishes between the declaration of a component dependency and the prerequisite for a component attribute. That is why Buckminster handles this special case by allowing to express an "obtained from" statement in the dependency declaration.

As an example suppose component A depends on component X, and component X is obtained by getting attribute X in component B. Then, the declaration in component A states the dependency on "component X obtained from component B's attribute X". The dependency on component B must also be specified.

With this construct in place, it is then possible to have prerequisites on attributes in the component X.

Versioning

Software has always been a challenging field because, unlike the continuous (or analog) nature of the real world, software is discrete (or digital). Thus a feature that worked in version 4.5 of the system may no longer work in version 5.0 or even in 4.6 or even in 4.5.1. A CSPEC always declares a dependency with reference to a particular component version. Actually, not just a single version. But let's look at the version information that Buckminster expects and how it is interpreted.

Component versions are referred to by a variety of terms. Although jumping ahead, the interpretation of version information will have a significant impact on how Buckminster will locate a component. When we are talking about component versioning in Buckminster, we will usually refer to the following terms. We will discuss them in detail in later sections.

• plain version string - This would be the commonly used way of referring to a version. Something like 3.2.1 or Titanic-1.0 would be typical examples.
• version designator - This is expressing a version range. You may want to say something like version "3.2.1 or later".
• version type - Buckminster knows about a number of version types which have an associated version string grammar (that is it tells you how a version string of a particular type should be structured).
• version selector - Buckminster gets components of a particular type. Depending on their location (i.e. a CVS or SVN repository) the version string will have to be translated in a path (fragment) allowing to access the component.

Normally, the version string would be something simple like "4.5". However, because Buckminster is designed for and by software developers, and because software is not developed in a linear fashion, the version information usually used in a dependency is more complex (and more useful) than the simple plain version string "4.5". Specifically, a version designator denoting a range is used. The characters '[' and ']' are used to denote an inclusive range and the characters '(' and ')' will denote an exclusive range. Here are some examples:

Example		Predicate
[1.2.3, 4.5.6)		1.2.3 <= x < 4.5.6
[1.2.3, 4.5.6]		1.2.3 <= x <= 4.5.6
(1.2.3, 4.5.6)		1.2.3 < x < 4.5.6
(1.2.3, 4.5.6]		1.2.3 < x <= 4.5.6
1.2.3		1.2.3 <= x

Examples of version ranges.

Buckminster is not limited to the major.minor.micro number notation used in this example. That notation is just one of several version types. The type interprets the plain version token(s) of a range and assigns a magnitude to the result so that versions can be compared. Buckminster allow news types to be added dynamically via its org.eclipse.buckminster.core.versionTypes extension point.

Mapping Versions to a Revision Control System

File:Comp A versioned dependencies.gif

The component DAG of our previous example with explicit branch specifiers on each dependency.

The versions used in the dependency specification will often be meaningless to a revision control system. Buckminster therefore uses a Version Converter.

It is the responsibility of the converter to create something that a particular revision control system can understand, and that will Buckminster eventually allow to get to the component. The converter is also able to convert in the other direction, and it thus plays a very important role when the best match for a version string is to be found in a revision control system. The plain version string is converted into a version selector, a Buckminster data structure describing if the version is a branch, or a tag, and/or it is based on a time-stamp or a change-number.

A combination of two version converters included with Buckminster and their functionality to use regular expressions makes it easy to set up even a quite complicated bi-directional mapping between a repository and a version selector. As an example, a plain version such as "3.1.0" can be converted into the version selector "main/v3_1_0-xyz" for a tag converter, or the plain version "3.1.0" into the version selector "three_one_zero/LATEST" for a branch converter. It is required that the mapping can be reversed without loss of information. See Version Selector for more information about the syntax of the data, and Version Converter for information about the two supplied converters.

The simplest version numbering (branch specifications) is the linear version numbers: 1.0, 1.1, 1.2, 2.0, etc. Linear version numbers are the base case: a revision control system with just one main branch.

Component B has one main branch, the <default> branch.

When the component dependencies have no branch specifier, Buckminster uses <default>/LATEST which translates to "the latest version of the default branch". The default branch varies by revision control system: CVS uses "HEAD", SVN uses "trunk", ClearCase uses "main", etc.

If the CSPEC dependency includes a version number, Buckminster uses a version converter to translate the version number into a branch specification. The most common translation is "version" into "<default>/version". For example:

Component A's dependency on component B is version "2.0" which gets translated to "<default>/2.0". Similarly, the dependency on component C is "1.1" which gets translated as "<default>/1.1". Looking at the branching diagram above, we see that "<default>" is "main", thus Buckminster will load "main/1.1" to satisfy that dependency.

By using a different version converter, Buckminster can map versions to change numbers, timestamps, or branches instead of tags. We expect the most typical configurations to be based on tags (something stable), or the latest on the main/default branch, or the latest on a particular branch. Combinations can sometimes make sense, but this also depends on the particular repository being used, and how content has been organized, tags set etc. Checkout Version Converter for more information.

What If There Isn't a Revision Control System?

What if the component is not stored in a revision control system? For example, what if the component is stored in an FTP directory ? The answer is that if the repository does not provide versions, then the Buckminster treats the repository as if it has just one version: <default>/LATEST. Requesting any other branch specification will fail. Or, to put the situation more positively, Buckminster treats a "no revision control system" situation as a simple revision control system that provides only one branch and one version.

What component languages are supported?

At the time of writing there are translators that can read component meta data and translate to CSPEC format on-the-fly for various Eclipse artifacts (including update sites), Maven, JARs and Buckminster assets. Buckminster can learn about new component languages. It's capabilities as a universal translator can be extended via two extension points:

• org.eclipse.buckminster.core.componentTypes - This extension makes Buckminster aware of a new component meta-data language associated with a component type. The implementation will facilitate the translation into a CSPEC. Currently supported component types and thus existing extensions are:
  • eclipse.project (expecting assets such as .project or plugin.xml files)
  • eclipse.installed (components that deployed in your Eclipse runtime)
  • jar (components that are JAR files)
  • maven (assuming Maven repository dependency information contained in a POM)
  • buckminster (essentially CSPEC, assumes that the owner of the asset has added a manually created CSPEC)
  • site.feature (update sites)
  • bom (a Bill of Materials artifact as produced by the Buckminster resolution process)
  • unknown (components for which no dependency information can be inferred or has been made available).
• org.eclipse.buckminster.core.readerTypes - A Buckminster Component Reader is also involved as it is responsible for the protocol used to get to the component data. Currently supported reader types and thus existing extensions are:
  • cvs (CVS repositories)
  • svn (Subversion repositories)
  • maven (Maven repositories; Maven 2 features are not fully supported yet)
  • p4 (Perforce repositories)
  • site.feature (Update sites)
  • eclipse.platform (installed target platform)
  • eclipse.import (replicating the Eclipse feature and plugin import)
  • url (single file URL; typically a jar, dll, or other pre-compiled artifact)
  • url.catalogue (folder or directory URL)
  • url.zipped (zip archive URL)
  • local (existing components, i.e. previously materialised).

Extended Component Specifications (CSPEXs)

Sometimes, the meta data in the component itself is not enough. This could be because the component meta data language does not have the expressive power to handle certain things that are important when materializing and performing component actions (compiling/building/packaging/installing/copying/encrypting or whatever). In these cases, Buckminster provides an extension mechanism to the CSPEC, called CSPEX.

The CSPEX extends the generated CSPEC with overrides and additions. The result is used by Buckminster.

The CSPEX, is stored in a file in the component (with the extension .cspex). The details are not within the scope of this introduction but it is important to know that it is in the same format as the CSPEC in general, but here it is also possible to override the generated specification.

Bills of Materials (BOMs) and Materialisation Specifications (MSPECs)

When Buckinster has finished the resolution of a configuration (i.e. when Buckminster has followed all prerequisites for an action, finding the correct version of other components via the stated dependencies) a Bill of Materials (BOM) is created. We called it a "component shopping list" earlier.

The BOM describes all the components and the version of each component included in the configuration (for full traceability, it also includes the Component Query that was used to produce the BOM). The materialization takes the BOM as input and materializes the required components (i.e. makes them available to the project).

By default the BOM is stored in Buckminster's internal model in memory, but it can also be stored externally as an XML artifact, thus allowing a materialization to be issued directly using the BOM itself. Instead of a CQUERY/RMAP, the BOM can thus be shared between team members as well. It should however be noted that the BOM represents a snapshot of your components taken at the time of resolution when the BOM was externalised from Buckminster's internal model. The same query, resolved later at a different time, may yield a different result (ie a different BOM) because certain components could have been moved in the interim, or newer versions may exist from the specified providers

Just as we were able to influence the component resolution process we are able to customize the materialization process. This is captured in a materialization specification (MSPEC) and is again an artifact that can be shared amongst team members. Via the MSPEC you can specify where you want components materialised. Initially, this may seem a pretty mundane feature, but looking beyond a workspace materialization it can have a significant impact in which location you materialize your components, in order to ensure a common starting ground for your team or simply be able to replicate exactly what someone else did before.