WTP CVS Restructuring

Purpose

In an effort to standardized groups on dev.eclipse.org especially in light of our recent WTP Projects Refactoring, the page is to propose and document how to restructure our CVS directories, so their access can be managed by associating directories with Unix Groups. In principle, we could leave all the directories the way they are now, and just carefully assign the Unix groups on a directory by directory bases ... but, this would be very complicated, error prone, and hard to scale in the future.

The document will start by focusing on the old wst and jst directories, as they need the most work, and other subprojects addressed later, towards the end of this proposal.

I have opened bug 207212 to track implementation and resolution of this issue. Please comment there if there are questions or suggestions for this document.

The Groups

First, I'll describe my best interpretation of what our standardized groups will be.

Code

The following are the Unix groups that will be associated with "code", in the cvs repository.

webtools.common-dev
webtools.datatools-dev
webtools.ejbtools-dev
webtools.jeetools-dev
webtools.servertools-dev
webtools.sourceediting-dev
webtools.webservices-dev
webtools.maps-dev
webtools.releng-dev
webtools.incubator-dev
webtools.jsf-dev *
webtools.dali-dev *
webtools.atf-dev *
* these are very similar in name to what they already are.

Website(s)

Per our WTP policy, all committers to any WTP project get full access to all of the WTP websites, so, we just need one group for that:

webtools.all    (currently, our "webtools" group is similar, 
                 but think webtools-home is used for this)

For reference, the current (cvs) website locations are

 org.eclipse/www/webtools (main one)
 org.eclipse/www/wtp      (incubator)
 org.eclipse/www/jsf      (used? do we still need this? need separate?)
 org.eclipse/www/dali     Used for redirect purposes; would like to keep the re-direct in place, but we should remove all of the content here
 org.eclipse/www/atf

Do we need new, separate ones for each of the new subprojects? (No, I don't think so).

Download site write access

This is currently limited to the "build team". The group is currently "webtoolsadmin" and I think the proposal for consistent naming is to use "webtools.downloads".

Map File Directories

Notice I'm proposing all the map file directories be moved to under one higher level "webtools.maps" directory. The purpose of that is, I'd like the map files to be writable by all, hence associated with webtools-all. Some of the divisions of the map files themselves can come after the main refactoring.

The reason is that occasionally someone from another project may have to releaes, or revert a particular map file from another project.

And, I checked, and there's no particular legal reason to "regulate" the map files ... they are similar to the web pages, there's no particular content to protect with IP rules, and I think there's little danger of someone accidently touching someone elses, unless there'd been prior agreement.

WST and JST to Refactored Projects

These initial WTP projects were created with the following sort of hierarchy -- abbreviated and idealized here to simplify explaining the principles of the restructuring. See the Detailed Listings section below the literal contents of the proposed new structure.

 wst
   components
      assembly
         features
         plugins
      server
         features
         plugins
         tests
      xml
         features
         plugins
            org.eclipse.wst.xml.core
            org.eclipse.wst.xml.ui
         tests
         examples
         development
         thirdparty
      xsd
         plugins
            org.eclipse.wst.xsd.core
            org.eclipse.wst.xsd.ui
         tests

The new proposed structure would look as follows

webtools.maps
   releng.common
   releng.sourceediting
   releng.servertools
   releng.webservices
   releng.datatools
   releng.jeetools
   releng.ejbtools
   releng.dali
   releng.jsf
webtools.releng
   features
   plugins
   org.eclipse.wtp.releng
   org.eclipse.wtp.releng.tests
   org.eclipse.wtp.releng.webupdatesite
   releng.builder
   releng.control
   releng.wtpbuilder
   releng.wtptools
   working
sourceediting
   features
   plugins
      org.eclipse.wst.xml.core
      org.eclipse.wst.xml.ui
      org.eclipse.wst.xsd.core
      org.eclipse.wst.xsd.ui
   tests
   examples
   development
   thirdparty
servertools
   features
   plugins
   tests

Principles

The above examples show a few points related to this restructuring.

1. The best top-level directory name should be similar to what the access group name will be: toplevel.subproject.component-subcomponent. We do not currently have components controlled by access restrictions, so I think we in webtools need only define the toplevel.subproject groups for now, hence the top level directories can be simply the subproject name, since 'webtools' is in its own cvs repository, so 'webtools' does not need to be part of the directory name. Also note that component and subcomponents can be added in the future, but best to start off with simplest, and add detail later, when needed.
2. Thus, everyone in a particular Eclipse subproject can be associated with a top level webtools directory, and automatically get access to everything in that directory, without the need for more complicated administration.
3. Note the previous "assembly" features and plugins will be become part of releng subproject. These are the Eclipse product plugins, and the features that define our builds. Each project will be responsible for their own features delivered to WTP (currently called the build-component features).
4. Note the example in source editing where what used to be directories, xml and xsd, are no longer distinguished by cvs directories, but all the plugins flattened under sourceediting/plugins.
   1. Conceptually and architecturally, xml and xsd could still be called 'components' but they do not have separate access control, so it's a bit overkill (and ultimately restrictive) to define them in separate cvs directories.
   2. Note that the "access control" for components such as xml and xsd is (still) social in nature. Legally, and in principle, anyone in the source editing project could change the code there, but in practice, there are individuals or small teams that normally do. Others, even in the same project, usually would attach a patch to a bugzilla if they wanted one of those experts to review and commit the code.
5. The map files will not have access restrictions per se (all committers will have access to 'webtools.maps' but the maps will be refactored in directories that contain map files only for that subproject.

Detailed Listings

The following are the detailed listing of directories in what would be their new structure. Hopefully the directory names, and groups names are obvious.

common

servertools

datatools

sourceediting

webservices

jeetools

ejbtools

org.eclipse.jsf

org.eclipse.jpa

org.eclipse.atf

webtools.releng

webtools.maps

To Do Items and Schedule

• 10/25 Proof on Concept

• try using CVS snapshots, on local, test machines with some move scripts to test to be sure we've accounted for everything.

Note: I've included some scripts in one of our cvs projects that work against a snapshot to do the reorganization. I'll see if those can be evolved to be the real things, to use when the real thing is done. See

org.eclipse.wtp/org.eclipse.wtp.releng.utilities

• Have Web Master sanity check the scripts and whole approach.

• 11/ 1 By the 11/1 Status meeting,

• Project leads to have reviewed the "detailed listings" for accuracy. Post requested revisions to wtp-releng, so revised listings can be created, preferably before the meeting.

• Releng (dw) to make sure all modules in all map files (for WTP 2.0 and 3.0) are listed in the modules file of CVS, so all (important) plugins show up in the flat listing.

• JSF, ATF, Dali should make a firm decision if they want to participate in re-org, to stay consistent, or, leave their directory structure as-is.

• 11/ 8 By the 11/8 Status meeting,

• spell out committer ID's for each Unix group

• make SED scripts to fix up map files, module files ... maybe project team sets?

• 11/ 13 By end-of-day

• Committers create patch files of any "unfinished" work that's not checked in, just for their personal backup.

• Committers check "unfinished" work into a temporary branch, if it's not ready for HEAD, so it will be moved during the mass move.

• Committers should create "Team Project Sets" of their favorite "in workspace" projects (assuming a SED script, or other editing can convert them).

• 11/ 16 1:00 PM - 5 PM Move day

• Begin the mass move

• Update modules file

• Update map files

• Do Test Build

• Do some quick Workspace tests

• Open the flood gates

JSF, JPA, and ATF

The directory-to-Unix-group associations will be pretty easy for these three subprojects and they can be left the way they are. Or, if any of those projects want to flatten their directory structure, this might be the time to do it. Each of those projects can provide their preference and will remain open for discussion, for now.

Reference and Notes

Committers can see our current CVS directories
/home/data/cvs/webtools

Snapshots of our CVS can be obtained at
http://archive.eclipse.org/arch/

There is a SED Script to "edit" old path/module names to new path/module names that can be found in one of our utility releng projects in cvs: org.eclipse.wtp.releng.utilities. It is named sedscript.sed