CBI/Feb21 2012

Meeting

• February 21, 2012, 10am EST
• Eclipse Conference facility

Attendees

Andrew R., Igor F., Paul W., David W.

Not on agenda, still being tracked as work left to do

 a) Platform specific code swt & launchers (Bug 370704)
 b) Do not generate new version qualifier unless there are actual code changes (Bug 367581)
 c) Isolate CBI platform build from uncontrolled software sources (Bug 369002)
 d) unexpected contents in org.eclipse.osgi.services sources bundles(Bug 370124)
 e) reproducible build version qualifiers (Bug 370707)
 f) API Analysis 
 g) Junit tests
 h) Which execution environments will CBI build & test against

Minutes

Discussed: Distribution wiki page

and Updated image w/ view of distribution

Andrew: I don't think a consensus was reached by those on the call. The format of these minutes will be informational and try to capture the essence of the important issues. The problem as described below seemed to be understood. There were questions as to whether Nexus (maven.eclipse.org) would help solve the problem.

Summarizing the discussion:

Problem statement

We need a system to manage 3rd party, modified 3rd party, and Eclipse-built artefacts that can support access/retention policies for the current release community, LTS, and Polarsys forges. It is likely that IWG member companies will want to establish their own company specific forge areas within the LTS & Polarsys forges. All of this would be replicated for each release every year.

Issues

1. Today this is done by hand via. cp/rsync, etc. This likely won't scale to satisfy our needs as LTS & Polarsys grow. As well, it may be increasingly error prone as we manage more and more software by hand.. [dw edit: ] As written, this sentence isn't true, or I misunderstand the intended context. We use b3 aggregator, which enhances some aspects of p2.mirror command to ensure a repo is self-consistent. Plain p2.mirror command is used (by many) to mirror p2 repositories either for more complicated cases, or simpler cases where they do not care if the result is self consistent. In some cases, people do use cp/rsync either because they think its easier for their case, or because it is a well understood, long used technology ... which does happen to work well if someone is just concerned about copying jars ... but will often be incorrect if trying to truly reproduce a p2 repository. Some projects use their own "composite repos", indirectly pointing to other repos, to have a "central place" to find the bundles or features they need. Nick Boldt mentions this in a 4 part series of blog entries of using p2 at an enterprise scale, starting with Part 1.
2. Finding components and navigating their dependencies makes for a barrier to those that wish to consume Eclipse software [dw edit: ] Again, I don't think this is accurate, as stated so generally, but, I am not sure what is intended. p2 makes if fairly easy to "finding and navigating dependencies" but perhaps this sentence is intended to be only about "those that do not use p2" (which, as I understand it) is a fairly small set of current Eclipse projects, and the use cases outside of Eclipse (as someone (Paul?) commented in the meeting) were thought to be fairly limited and special purpose ... such as those using swt or emf in a stand-alone java application and some of those happen to use maven ... others use apache ivy .. but the vast majority of Eclipse features and bundles can not be used in standalone "java app" so there would be little value to putting "everything at Eclipse" in a maven repository. As I understand it (and, I could be wrong).
3. Today, some projects (e.g. Virgo, Stardust, possibly others) are consuming software from maven central. This is not in compliance to Eclipse's IP policy.
4. Leaving this for each project to sort out on their own will inevitably result in redundancy and diverse solutions. A unifying solution lets projects focus their precious time on their project rather than a common distribution problem.

Relevant information

1. p2 and Maven repository formats are different and incompatible. It is believed converting from one to the other inevitably loses information. [dw edit: ] bug 371475 discusses many of the problems between maven artifacts and OSGi artifacts. (I think its not just "p2 vs. Maven", but as much or more "OSGi vs Maven".)
2. Nexus 2, under the EPL license, can import a zip file containing a p2 repository. This is just held however, and served at a URL. It doesn't enable/tie into Maven dependency calculations, etc.
3. [dw edit: ] One point made at the meeting (I think by Paul) was that we have been releasing software for 10 years, and I'll add Simultaneous Releases for 6, and during that time hundreds? of companies have been building hundreds of products on top of Eclipse. So, while that's not to say improvements can not be made, it doesn't seem fair to say it is all that bad the way it is. It seems the "tone" of some of these notes and meetings is that radical changes are needed because we are failing. I think it'd be more constructive for those with "special cases" that are not well served by current system detail those use cases, and they work on incremental changes that better suits their use cases, and once that's understood, generalize from there, rather than overhaul what exists based on trying to be all things to all people. Naturally, I'm biased, since I'm used to the current system, ... but, wanted to document the counter point of view.

Questions

1. Should it be a requirement that eclipse builds produce maven repositories in addition to p2 repositories?
2. What work is required to enable nexus to be a suitable platform for distributing p2 repositories? Nexus 2.0 looks like it can do most of this already.

Proposal for anonymous forums replies

[dw edit] A side topic, perhaps, but it was mentioned during the meeting that it was being considered to have or allow "anonymous comments" in forum or mailing lists. As far as I know, this has not been done before at Eclipse and I think deserves broad discussion. I personally think there are pros and cons.

I have heard of this kind of "anonymous commenting" has the positive effect of "leveling" corporate structures, so "junior members" do not feel intimidated to contradict a more "senior person" (or, maybe their own manager! :) if everyone comments anonymously.

I've also heard it is an important principle when people are disusing political-type issues, since it helps insures "free speech".

So, I do see those advantages, and I think perhaps Eclipse would benefit from it in some contexts or some conversations, but it does also, to me, seem kind of counter to the principles of Eclipse's meritocracy, transparency, etc., in many other contexts, so I am just documenting this note from the meeting to ensure it gets broad discussion and definition.

Comment on who participates and who does not

[dw edit] Probably a side topic, but it was stated during the meeting that projects do not need to "buy in" into being in LTS and need not be part of the yearly Simultaneous Release ... that it is up to the Steering Committee on which projects to include or not depending on their own needs or criteria. (If I understood correctly). I just note it since it may effect some mechanisms or policies or goals. (It may not ... I just thought it worth clarifying).

Actions

1. David, do you mind providing a link to the discussion about what Jetty did? You mentioned something about them creating their own repository.

There was a long discussion on orbit-dev list, concluding with "Jesse's solutions", at msg02398 ... but the dozen or so previous message in the chain should be read as well, as many contain relevant information and history. This most recently came up in bug 371396 which, to me, showed there is a lot of confusion about what "maven.eclipse.org" is intended to solve ... it should not be used to change a project's retention policy (Orbit, and other projects, must reserve the right to remove or change bundles during milestone phases, and it would be wrong to "keep" those in maven.eclipse.org (as it would allow use of "non released" bundles). Plus, seems to be confused with the long standing "single URL" problem that is about p2, not maven, which is described in bug 289092 and can be solved without anything to do with maven.