COSMOS Design 204921: Service Group in Broker i8

Service Group in Broker i8

Change History

Name:	Date:	Revised Sections:
Hubert Leung	12/10/2007	Initial version

Overview

The Broker is the component in the COSMOS data collection framework that holds a registry of Data Managers and allows clients to query for addresses based on some constraints. In a system supporting COSMOS and CMDBf, Management Data Repositories (MDR) and federating CMDB are speialized instances of data managers, and will also be registered at the broker.

In i7, we have an implementation of a data broker that makes use of an XML file to store the EPRs and property values (such as classification, dialect and name) of data managers, and we support querying ERPs by classification value. We will improve the implementation in i8 by using web service standards as defined in WS-ResourceFramework (WSRF) and WS-DistributedManagement (WSDM) in the broker for the storage and retrival of properties of data managers.

Web Services Service Groups

The service group specification defines a method for grouping web services. This specification is very suitable for the implementation of the COSMOS broker as a registry of data managers, which are exposed as WSDM endpoints.

Specifically, here are a few quotes from the service group specification for defining the purpose and scope of service groups:

"This WS-ServiceGroup specification defines a means by which Web services and WS-Resources can be aggregated or grouped together for a domain specific purpose. In order for requestors to form meaningful queries against the contents of the ServiceGroup, membership in the group must be constrained in some fashion. The constraints for membership are expressed by intension using a classification mechanism. Further, the members of each intension must share a common set of information over which queries can be expressed."

"[...] specialized interfaces may offer means of querying the contents of the ServiceGroup, and for performing collective operations across members of the ServiceGroup."

Here are some benefits of using service groups:

• Standard APIs
• Persistence
  • Apache MUSE provides an implementation of service groups, which includes a file-based serialization implementation of group entries.
  • It is possible to use an alternative persistence mechanism (e.g. database), simply by modifying the muse.xml to point to another persistence implementation class, which can be provided by the adopters.
• Configurable: The service group can be configured to require a set of mandatory properties by member services (data managers). These properties can be used as query contraints when querying for EPRs.

Use Cases

Use Case 1: Configure broker service group

• Configure some properties of the broker for a specific deployment of the broker. It's expected that the configuration is rather static and is done before the first data manager registers with the broker.
• Configurable properties include:
  • MembershipContentRule: resource properties required by member data managers
  • Persistence:
    • Turn persistence on/off
    • Provide alternative persistence mechanism (MUSE provides a file-based persistence)
• Some tooling can be provided to facilitate this step. With the help of tooling, the implementation aspect of WSDM and MUSE implementation can be shielded from the deployers, who wants to focus on the business meanings of these configuration values. Tooling is not out of scope in this enhancement.

Examples

• Configure the service group to require property ManageabilityCapability (Defined in http://docs.oasis-open.org/wsdm/muws1-2.xsd) from all data managers.
• Other useful properties defined by MSDM that can be used to indexing or searching for data manager include ResourceId, Caption and Description.
• The property can also be user-defined (i.e. not part of a standard). For example, if we want to support the concept of "classification" to categorize data managers as in i7 and earlier, we can specify a "classification" property with namespace http://www.eclipse.org/cosmos/dataManager, and all data manager need to have a managed property with the name "classification".

Use Case 2: Register data manager with broker

• Data manager registers with broker on start up, or by executing an operation defined by the data manager.
• Data manager invokes the add API defined by the service group registration capability, providing the data manager EPR as a parameter.
• The broker service group creates a client to the data manager using the ERP, and retrieves the values required by the service group using the GET capability defined in WS-RP standard. If the data manager does not support a required property, the registration will fail. Otherwise, the EPR along with the property values are persisted in a file. (The operations in this step is provided by the MUSE implementation.)

Use Case 3: Query broker for data manager EPRs with constraints

• Query for data managers providing a property name in the form of a qualified (QName) and a value.
• If the broker does not support the property, the operation should fault.
• Broker gets the resource property document, which includes all member EPRs and property values, selects entries that satisfy the value provided in the query (an equality match), and return the EPRs to the client.
• The proxy client should return the data managers in the form as an array of WsResourceClient objects. The client can get all properties that are exposed of managed properties using the WsResourceClient API.
• (Alternative implmentation) Return the an Element array of <wsrf-sg:Entry> elements. The entries are selected from the property document based on the query criteria. This approach will offer better performance since no extra calls are required to get the property values of data managers. However, convenience methods for parsing the XML fragment is required.
• Return all members if the value in the query is null.

Examples

• If broker require data managers to support "ManageabilityCapability", the client can do ask a question like "Give me all data managers that support the CMDBf Query Capability." or "Give me all data managers that support the SDMX Query Capability". The use of ManageabilityCapability is similar to the notion of "dialect" as in i7 and earlier. It is for finding Data managers that talk a certain language.
• Sometime ManageabilityCapability is not sufficient to produce a concise list of data managers. It may be desirable to also query by "Caption" (display name), or other properties.

Implementation Details

Persistence

• To enable persistence, edit muse.xml, and add

use-router-persistence="true"

to both service group and service group entries resources.

• Specify persistence implementation class in service group capability declaration (also in muse.xml):

 <capability>
        <capability-uri>http://docs.oasis-open.org/wsrf/sgw-2/ServiceGroup</capability-uri>
        <java-capability-class>org.eclipse.cosmos.dc.data.broker.CustomSimpleServiceGroup</java-capability-class>
	<persistence>
	    <java-persistence-class>org.apache.muse.ws.resource.sg.impl.ServiceGroupFilePersistence</java-persistence-class>
	    <persistence-location>service-group-entries</persistence-location>
	</persistence>
 </capability>

• Known problems:
  • There is problem when reloading resources from file when restarting broker, need to add set validate-wsrp-schema flag to false to mask the problem.
  • There is a bug in the MUSE implementation that prevented the persistence of service group member. A workaround is provided by using a providing a customized service group implementation.
• Here is an example of an XML file of a service group member in the file-based registry:

<?xml version="1.0" encoding="UTF-8"?>
<wsrf-sg:Entry xmlns:wsrf-sg="http://docs.oasis-open.org/wsrf/sg-2">
    <wsrf-sg:ServiceGroupEntryEPR>
        <wsa:Address xmlns:wsa="http://www.w3.org/2005/08/addressing">http://localhost:8080/broker6/services/service-group-entry</wsa:Address>
        <wsa:ReferenceParameters xmlns:wsa="http://www.w3.org/2005/08/addressing">
            <muse-wsa:ResourceId xmlns:muse-wsa="http://ws.apache.org/muse/addressing">
                 uuid:ca56d697-4398-349a-a6a6-8f0b6c4508f0</muse-wsa:ResourceId>
        </wsa:ReferenceParameters>
    </wsrf-sg:ServiceGroupEntryEPR>
    <wsrf-sg:MemberServiceEPR>
        <wsa:Address xmlns:wsa="http://www.w3.org/2005/08/addressing">
            http://localhost:80/cosmos/services/org.eclipse.cosmos.example.mdr.ExampleMdr</wsa:Address>
        <wsa:ReferenceParameters xmlns:wsa="http://www.w3.org/2005/08/addressing">
            <muse-wsa:ResourceId xmlns:muse-wsa="http://ws.apache.org/muse/addressing">
                ExampleMdr</muse-wsa:ResourceId>
        </wsa:ReferenceParameters>
    </wsrf-sg:MemberServiceEPR>
    <wsrf-sg:Content>
        <myns:ManageabilityCapability 
           xmlns:myns="http://docs.oasis-open.org/wsdm/muws1-2.xsd">
              http://docs.oasis-open.org/wsdm/muws/capabilities/Identity
        </myns:ManageabilityCapability>
        <myns:ManageabilityCapability 
           xmlns:myns="http://docs.oasis-open.org/wsdm/muws1-2.xsd">
              http://docs.oasis-open.org/wsdm/muws/capabilities/ManageabilityCharacteristics
        </myns:ManageabilityCapability>
        <myns:ManageabilityCapability  
           xmlns:myns="http://docs.oasis-open.org/wsdm/muws1-2.xsd">
              http://docs.oasis-open.org/wsdm/muws/capabilities/Description
        </myns:ManageabilityCapability>
        <myns:ManageabilityCapability 
           xmlns:myns="http://docs.oasis-open.org/wsdm/muws1-2.xsd">
              http://docs.oasis-open.org/wsdm/muws/capabilities/OperationalStatus
        </myns:ManageabilityCapability>
    </wsrf-sg:Content>
</wsrf-sg:Entry>

• There will be one XML file created for each group member. The XML files are located at WEB-INF\classes\service-group-entries\service-group-entry. There will also be entries in WEB-INF\classes\router-entries\service-group-entry for the EPR of the service group entry resource.

Configuring Membership Rules

• Need the following section in the RMD file, in WEB-INF\classes\wsdl directory. Note how the property is specified in the StaticValues element.

<Property name="wsrf-sg:MembershipContentRule"
	modifiability="read-only" mutability="mutable">
	<StaticValues>
		<wsrf-sg:MembershipContentRule
			ContentElements="myns:ManageabilityCapability" 
			xmlns:myns="http://docs.oasis-open.org/wsdm/muws1-2.xsd"/>
	</StaticValues>
</Property>

<Property name="wsrf-sg:Entry" modifiability="read-only"
	mutability="mutable" />

Test Coverage

TBD

Task Breakdown

• Data Manager
  • use the new registration mechanism
  • Expose properties as managed properties
• Broker server
  • Change implementation of getDataManger operation to query on service group
  • The registerDataManager API can be removed, because we will use the ServiceGroupRegistration standard (web service) API to do registration.
  • Remove implementation of the XML broker registry
• Broker client
  • provide proxy API to make query

Related Links

Broker design doc for i7

Apache Muse - Add WSRF Service Groups to an Endpoint

Service Group Specification

Open Issues/Questions

Please provide feedback in the talk page.