COSMOS Design 208274

Create a data manager toolkit that will allow adopters to easily register and use a data provider inside COSMOS framework

This is the design document for bugzilla 208274.

Change History

Name:	Date:	Revised Sections:
Mark Weitzel and Hubert Leung	11/7/2007	Initial version
Martin Simmonds	1/18/2008	Initial design details of project wizard
David Whiteman	1/22/2008	Added rest of design template Added screen mockups
David Whiteman	1/24/2008	edits per MDW feedback
David Whiteman	1/25/2008	edits per Ali feedback
David Whiteman	1/29/2008	edits per design review feedback
David Whiteman, Hubert Leung	2/1/2008	Added J2EE vs. OSGi design strategy information

Workload Estimation

Rough workload estimate in person weeks

Process	Sizing	Names of people doing the work
Design	1	David (initial work by Hubert/Martin/Mark)
Code	2	David/Hubert
Test	1	David/Hubert/QA
Documentation	2	David/Rich
Build and infrastructure	0
Code review, etc.*	0
TOTAL	6.0

Terminologies/Acronyms

The terminologies/acronyms below can be used throughout this document.

Note: These definitions are not meant to be general definitions of these terms, but rather refer to how the terms are used within this design document as applied to COSMOS.

Term	Definition
CMDB	Configuration Management Database as defined by ITIL
CMDBf	CMDB Federation Specification
MDR	Management Data Repository as defined by CMDBf
Federating CMDB	As defined by the CMDBf spec, a CMDB that federates data from multiple MDRs
CMDBf toolkit	The code provided by COSMOS that interacts with the query and registration service framework, and provides the WSDM endpoints needed to communicate with the federating CMDB. It also includes the Eclipse tooling specified in this design.

Purpose

There are a number of manual steps that adopters are required to take when creating a data manager, MDR, or federating CMDB using the COSMOS framework. Much of these steps can be automated by an Eclipse-based toolkit that will greatly simplify the steps required for an adopter to use COSMOS framework. The toolkit should automatically generate code stubs, configuration files, and any other artifacts that will greatly simplify the procedure for adding a custom data manager to COSMOS. It should also allow for easy testing of data managers from within the Eclipse environment.

Use Cases

The following have been identified as use cases related to this enhancement.

1. Developer creates new data manager
   There are two variations on this use case:
   1. Data manager is being created from scratch.
      This is addressed in Creating a new Data Manager project.
   2. Data manager is being adapted to existing data source.
      This is the case where the user imports an MDR into Eclipse. Users will create a normal Eclipse project, and then based on the config.properties and domainEPR.xml files, new manifest editor pages will be displayed for editing these. See Updating Data Manager Project.
2. Developer updates properties of data manager
   This is addressed in Updating Data Manager Project.
3. Developer wants to start data manager in Eclipse.
   This is addressed in Deployment.

Development environment

• Developer will install a set of plug-ins into the Eclipse workbench, and will download noted COSMOS prerequisites. If J2EE deployment is desired, the additional prerequisite of [WTP] will need to be installed. The plug-ins will be packaged such that if only OSGi deployment is desired, WTP will not be needed.
• Developers can test toolkit implementation using WSDM Tooling, as it allows the launching of endpoints within Eclipse
• Developer will export the end result into deployment environment:
  • J2EE, OSGi
  • Server component
  • Client component
• Documentation on how to build data managers will be provided, and integrated with the Eclipse help system

Extensibility

Adopters will create extensions to the COSMOS framework in the Eclipse workbench environment. The following extension points will be provided:

• org.eclipse.cosmos.me.dmtoolkit.dmtypes - List of specialized data manager types (e.g. MDR, federating CMDB). List of project creation wizard pages for specialized data manager types to enter custom properties (TBD).
  Example usage:

  <extension
      point="org.eclipse.cosmos.me.dmtoolkit.dmtypes">
      <dataManagerType
          id="org.eclipse.cosmos.me.dmtoolkit.dmtypes.mdr"
          name="MDR"
          description="Management Data Repository"
          icon="resources/mdr.gif"
          extends="org.eclipse.cosmos.dc.mdr.impl.AbstractMdr"
          requiredPlugin="org.eclipse.cosmos.dc.mdr"/>
  </extension>
  

• org.eclipse.cosmos.me.dmtoolkit.pkgtypes - List of packaging types (e.g. J2EE, OSGi). List of Export wizard pages for packaging type to enter custom properties.
• org.eclipse.cosmos.me.dmtoolkit.mapping - List of allowed data manager types for a given packaging type

Examples and documentation on how to extend the COSMOS framework will be provided, integrated with the Eclipse help system.

Detailed Design

The code for this enhancement will be part of the Management Enablement project. The code will span multiple plug-ins. The primary target for this iteration is J2EE deployment, and OSGi deployment is a stretch goal that can be deferred to i10 if necessary. The plug-ins will be structured as follows:

• org.eclipse.cosmos.me.dmtoolkit.common - contains code common to both J2EE and OSGi deployment scenarios
• org.eclipse.cosmos.me.dmtoolkit.j2ee - contains code specific to the J2EE deployment scenario. This plug-in will have a dependency on the Eclipse Web Tools Project (WTP).
• org.eclipse.cosmos.me.dmtoolkit.osgi - contains code specific to the OSGi deployment scenario. This will be done last in case we cannot contain this for i9.

UI workflows

The following sections describe the new UI workflows. The individual wizard pages will be presented the same in both J2EE and OSGi scenarios.

Creating a new Data Manager project (J2EE deployment)

In the J2EE scenario, the user would create a Dynamic Web Project in WTP, and would select facets that essentially provide "feature packs" for providing the functionality of custom wizard pages, generated code, and additional runtime requirements. This saves the need for subclassing the WTP wizards and allows for a more flexible design. The approach is explained well in this Eclipse.org tutorial article.

The custom wizard pages added via the facets will have the same layout as those described in the OSGi section following.

Creating a new Data Manager project (OSGi deployment)

We will provide a new Eclipse menu option for creating a data manager project. As mentioned previously, this will greatly simplify the process and reduce the human error in configuring a data manager or MDR for use with COSMOS.

The menu option will be accessible from the Eclipse menu bar, as follows:

Create Data Manager Project Wizard

The wizard for creating a data manager project will have the following responsibilities. Note: The *mdrname variable in the package names will use the value of the Data manager name field, in lowercase form, with spaces removed. The *userpackageprefix variable is a value supplied by the user to indicate the prefix for all generated package names.

• Creates projects:
  • *userpackageprefix.*mdrname.dataManager.server
  • *userpackageprefix.*mdrname.dataManager.client (if client option selected)
• Creates Activator.java
• Creates *mdrnameDataManager.java
• Creates *mdrnameDataManagerClient.java (if client option selected)
• Create config.proprties
• Create domainEPR.xml
• Create MANIFEST.MF
• Creates Plug-in Dependencies
  • org.apache.xerces_2.8.0*
  • org.apache.xml.resolver*
  • org.apache.muse.complete
  • org.eclipse.osgi_3.3.2.R33x* (only when deployment target is OSGi)
  • org.eclipse.cosmos.common
  • org.eclipse.cosmos.dc.broker.client
  • org.eclipse.cosmos.dc.management.domain.client
  • org.eclipse.cosmos.dc.dataManager (only when data manager type is standard data manager)
  • org.eclipse.cosmos.dc.mdr (only when data manager type is MDR)
  • org.eclipse.cosmos.dc.federating.cmdb (only when data manager type is Federating CMDB)
  • CMDBfCommon.jar (when data manager type is MDR or Federating CMDB)
  • CMDBfQueryTransformation.jar (when data manager type is MDR or Federating CMDB)
  • CMDBfQueryService.jar (when data manager type is MDR or Federating CMDB)
  • CMDBfRegistrationTransformation.jar (only when data manager type is Federating CMDB)
  • CMDBfRegistrationService.jar (only when data manager type is Federating CMDB)
  • CMDBfMetadata.jar (only when data manager type is Federating CMDB)
  • org.eclipse.cosmos.me.management.common
  • org.eclipse.cosmos.me.management.wsdm
• Creates Plug-in Dependencies (Client Project)
  • org.eclipse.cosmos.dc.dataManager.client (data manager type = standard data manager)
  • org.eclipse.cosmos.dc.mdr.client (data manager type = MDR)
  • org.eclipse.cosmos.dc.federating.cmdb.client (data manager type = Federating CMDB)

When the 'Create Data Manager' project wizard is launched, properties general to all data managers can be set on the first page:

1. MDR Name (used for name in manifest and generated project/package/class names; denoted by *mdrname elsewhere in this design)
2. Prefix for generated plug-in and package names (denoted by *userpackageprefix elsewhere in this design)
3. Resource_ID
4. Display Name
5. Description
6. Whether to create a client-side plug-in

Here is a mockup of the appearance of the wizard:

Create client plug-in option

When the Create client plug-in option is selected, the user will be prompted for a custom class to be instantiated for the client. If Use default client is selected, the default implementation for the selected data manager type (e.g. MdrClient) will be used.

Generating EPRs

Building EPRs by hand is a tedious and error prone activity. This is a place where UIs can be really helpful, since you can have integrated help, lists of valid values, and more readable labels. Plus, typing raw XML is laborious. This section defines a UI so users don't have to hand-code these EPRs. Here is a mockup for the input controls for generating an EPR, which will be on page 2 of the project creation wizard.

Deployment target pages

Another extension point will allow adopters to provide custom wizard pages for the desired deployment target. These pages will allow input of target-specific properties.

Data manager type selection

A wizard page will be displayed to prompt the developer for the type of data manager to be created. There will be a new extension point to supply the list of data manager types; the default implementation of the extension point will define the values "Standard data manager", "Management data repository (MDR)", and "Federating CMDB". For each type, other information such as icon, id, description, abstract parent class, and prerequisite plug-in will be indicated in the extension point. The icon, name, and description will be displayed in the page.

There will be yet another extension point that specifies which data manager types are allowed for the desired deployment target (i.e. J2EE or OSGi); this subset will be presented for user selection.

Another extension point will allow adopters to provide custom wizard pages for the selected data manager type. These will allow the input of properties unique to the data manager type.

Updating Data Manager Project

A valid use case is for a developer to want to change some of the initial project settings after the project has been created. We will provide an extra page on the project manifest that will allow several of the project parameters to be changed. We will need to be sure to preserve user edits to the project artifacts that are made outside our editors.

Data Manager Project Artifacts

The following examples illustrate the artifacts that would be generated if a data manager of type "MDR" is specified in the project wizard.

Activator.java

package org.eclipse.cosmos.*mdrname.dataManager;

import org.eclipse.cosmos.dc.dataManager.api.IDataManager;
import org.eclipse.cosmos.dc.dataManager.impl.AbstractDataManagerActivator;

public class Activator extends AbstractDataManagerActivator {

	@Override
	protected IDataManager getDataManagerInstance() {
		return new *MdrnameDataManager();
	}

}

*MdrnameDataManager.java

The parent class of the data manager class depends on the data manager type:

• standard data manager: AbstractDataManager
• MDR: AbstractMdr
• Federating CMDB: AbstractFederatingCMDB (ref: COSMOS_Design_215267)

package org.eclipse.cosmos.*mdrname.dataManager;

import org.eclipse.cosmos.dc.cmdbf.services.query.service.IQueryHandlerFactory;
import org.eclipse.cosmos.dc.dataManager.api.IDataManager;
import org.eclipse.cosmos.dc.mdr.api.IMdrQuery;
import org.eclipse.cosmos.dc.mdr.impl.AbstractMdr;
import org.eclipse.cosmos.dc.mgmt.annotations.ManagedResource;
import org.eclipse.cosmos.samples.cmdbf.services.query.ICMDBfSampleConstants;
import org.eclipse.cosmos.samples.cmdbf.services.query.QueryHandlerFactory;
import org.eclipse.cosmos.samples.cmdbf.services.query.XMLRepository;

@ManagedResource
public class *MdrnameDataManager extends AbstractMdr implements IDataManager, IMdrQuery 
{	
	public *MdrnameDataManager() 
	{
	}

	@Override
	public IQueryHandlerFactory getQueryHandlerFactory() 
	{
		return QueryHandlerFactory.getInstance();
	}

}

config.properties

The contents of this file are built from the values entered on the first page of the project creation wizard. This file will also be updated when you change these values using the project editor wizard, which is opened when you edit this file in Eclipse.

RESOURCE_ID=Example
DISPLAY_NAME=MDR Example
DESCRIPTION=An example of an MDR
MGMT_DOMAIN_EPR_FILE=META-INF/domainEPR.xml

domainEPR.xml

The contents of this file depend on the values entered on the Endpoint Reference page of the Project Creation Wizard. This file will also be updated when you change these values using the project editor wizard, which is opened when you edit this file in Eclipse.

<?xml version="1.0" encoding="UTF-8"?>
<wsa:EndpointReference xmlns:wsa="http://www.w3.org/2005/08/addressing">
    <wsa:Address>http://localhost:8080/cosmos/services/domain</wsa:Address>
</wsa:EndpointReference>

web.xml (J2EE)

The web.xml file from the web project needs to be modified to include the following, setting the initializer class as appropriate:

        <listener>
		<listener-class>
		org.eclipse.cosmos.example.mdr.ExampleMdrInitializer
		</listener-class>
	</listener>	

 	<servlet>
		<display-name>Apache Muse Servlet</display-name>
		<servlet-name>ApacheMuseServlet</servlet-name>
		<servlet-class>org.apache.muse.core.platform.mini.MiniServlet</servlet-class>
	</servlet>
	<servlet-mapping>
		<servlet-name>ApacheMuseServlet</servlet-name>
		<url-pattern>/*</url-pattern>
	</servlet-mapping>	
 	<context-param>
		<param-name>CosmosWebPort</param-name>
		<param-value>8080</param-value>
	</context-param>

MANIFEST.MF (OSGi?)

• The Bundle-Name is set using the Data Manager Name value
• The *userpackageprefix variable in the package name will be a user-prompted value used for the prefix on all generated packages
• The *mdrname variable in the package names will use the value of the "Data manager name" field, in lowercase form, with spaces removed

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: Example MDR
Bundle-SymbolicName: *userpackageprefix.*mdrname.mdr
Bundle-Version: 1.0.0
Bundle-Activator: *userpackageprefix.*mdrname.mdr.Activator
Import-Package: org.osgi.framework;version="1.3.0"
Require-Bundle: org.eclipse.cosmos.dc.mdr,
 org.apache.xerces,
 org.eclipse.cosmos.common,
 org.eclipse.cosmos.dc.cmdbf.services,
 org.apache.muse.complete,
 org.eclipse.cosmos.samples.cmdbf.services
Export-Package: *userpackageprefix.*mdrname.mdr

Import Wizard

A developer will want to be able to import an existing data manager into Eclipse. We will need to provide an import wizard (using the Eclipse import dialog extension point) for the case where the data manager is not being built from scratch. The developer would select File > Import... from the Eclipse menu bar to navigate to this wizard.

Testing (J2EE)

Because the generated project is a WTP dynamic web project, the developer can simply choose the Run on Server option to launch the data manager.

Deployment (J2EE)

The plugins will be exported using the standard Eclipse plugin export dialog, with the format of WAR specified.

Testing (OSGi)

We will provide a context menu option for data manager projects as follows: Systems Management > Deploy.... This will allow the developer to run the data manager within Eclipse. The problem with using the standard Run... menu option is there are multiple, tedious steps involved in setting up the launch configuration. See COSMOS_Programming_Model#Running_the_MDR_within_Eclipse for an example of those manual steps. The new menu option will ensure all of these steps happen under the covers.

Deployment (OSGi)

This approach is TBD.

Task Breakdown

Required tasks:

1. WTP facets created
2. Deployment type extension point created
3. Data manager type extension point created
4. Data manager project wizard properties page created
5. Data manager project wizard EPR page created
6. Data manager project wizard deployment target page created
7. Data manager project wizard data manager type page created
8. Manifest editor created for config.properties
9. Manifest editor created for domainEPR.xml
10. TPTP manual tests written, and automated GUI testing invesigated
11. Import wizard created for existing data managers (?)

OSGi related tasks that could be deferred:

1. Data manager properties file artifact created
2. Data manager project builder created
3. JUnit test written to ensure builder is functional
4. OSGi data manager project wizard created
5. OSGi deploy action on data manager project created
6. JUnit test written to test deploy action

Test Coverage

JUnit tests will be provided for the following scenarios:

1. A data manager project builder will be given a data manager properties object to execute. The test will assert the project was created with the desired characteristics. Variations will be tested for data manager type, deployment type, and other properties called out in this design. Invalid values, such as an existing project name, will be tested.
2. The deploy action on a data manager project will be invoked. The test will confirm that this action results in a running data manager.

TPTP manual tests will be provided to test the following:

1. Basic end-to-end test: create standard data manager via creation wizard and attempt to deploy the data manager
2. Create data manager project wizard: Ensure that the correct set of allowable data manager types are presented after a deployment target is selected

Open Issues/Questions

All reviewer feedback should go in the Talk page for 208274.

References

• COSMOS Programming Model
• Providing a CMDBf Query and Registration Service