COSMOS Design 231343

Improve query capability of the COSMOS broker

Change History

Name:	Date:	Revised Sections:
Hubert Leung	June 23, 2008	Initial version

Workload Estimation

Rough workload estimate in person weeks

Process	Sizing	Names of people doing the work
Design	0.2	Hubert Leung
Code	0.8	Bill Muldoon
Test	0.4	Bill Muldoon
Documentation	0.2	Bill Muldoon
Build and infrastructure	0	Saurabh Dravid
Code review, etc.*
TOTAL	1.6

'* - includes other committer work (e.g. check-in, contribution tracking)

Purpose

Allow the COSMOS broker to accept more specific search criteria when searching for a list of data managers. The goal is to return a more relevant list of EPRs of data managers.

Requirements

In the current implementation (as of i11), the broker supports three query operations:

1. Get all data managers: return the list of all data managers registered at the broker
2. Get data managers by hostname and name
3. Get data managers that support a given web service: Provide the namespace of a web service definition as a parameter. Return the list of all data managers that support a web service with that namespace. E.g. "Find all data managers that support the CMDBf query service" will return the list of all MDRs.

Limitations:

1. Searching for data managers by service namespace is sometimes insufficient. For example, when searching for MDRs, we want to filter by the record types managed by the MDRs. "Find all MDRs that manages asset data." Simply asking for data managers that support the CMDBf query service is not specific enough, and the query result may include EPRs of data managers that manage irrelevant data.
2. Cannot support for data managers by new configuration values introduced by enhancement 233690.

New query capabilities to be supported:

1. query EPRs by ID of data manager
   • use case: a client queries the federating CMDB for a list of configuration items. Each item will have an InstanceID element which include the mdrId and the localId. The client can find the EPR the MDR from the broker by providing the MDR ID. With the EPR of the MDR, the client can issue CMDBf queries to the MDR.
2. query by property values of the data manager: return data managers that has the required properties name-value pairs
3. query by data-namespace of a CMDBf query service: return data managers that support the CMDBf query service, which indicates the given namespace in the service metadata for the recordTypes.
   • Address limitation 1 explained above.
   • use case: The current implementation allows the broker client to query for data managers by the namespace of the web services supported by the data managers. In CMDBf scenario, all MDRs support the CMDBf query service. However, we need more information to differentiate between the MDRs as they provide different information. We want to avoid asking the broker client to get a long list of MDRs and loop through them to decide which one in fact contains the type of data the client application is looking for. The service metadata of the CMDBf query service can indicate the namespace of record types. This new function is to allow broker client query for MDRs by the data-type namespace. The namespace is the unique identifier of the schema that defines the data structure.
4. Remove the get data managers by hostname and group name function: enhancement 233690 has proposed to consolidate the indentifiers of all data managers to use globally unique URIs.

Use case

Use Case: Query for a list of MDR’s and their Meta Data

Design

Cache more static values of the data managers at the broker, and allow the use of these values as query criteria when querying for EPRs of data managers.

Design considerations

1. Broker clients can only query for EPRs of data managers by values cached at the broker. The broker won't dispatch queries to multiple queries to do the filter matching at real time.
   • The indexes of data managers are static values, and can be cached at the broker
   • Better performance
   • Reduce complexity
2. Queries for data managers will only return EPRs, not metadata definition.
   • Metadata can be retrived from the client of the data managers. We don't need to use the broker to relay the information.
   • Note that this decision is contrary to the original description of this enhancement request, but I think the solution can address the use case.
3. Broker only stores static values about data managers.
4. Broker only stores locale-independent data.
5. Use namespaces to identify data model and schema instead of keywords.
   • The use case written on the use case page asks for a function to search EPRs by keywords such as "storage". You can consider a data schema as a way to define a dictionary of "keywords" and the data structure. Using the namespace to identify a schema can effectively gaurentees the datastore supports a certain set of data types and data structures.
   • The namespace of record types can be specified in the service metadata of CMDBf query and registation services.

Registering data managers with broker

• During registration, the following values will be included for each data manager, in addition to the current values:
  1. ID (an URI, equals to the MDR ID for an MDR)
  2. RecordTypeNamespace (for CMDBf query and registration services, if defined in service metadata)

New web service operations for the broker

1. getDataManager

Pseudo-schema for the SOAP body:

<getDataManager>
  <id>xs:anyURI</id> |
  (<property name="xs:string" value ="xs:string"> *
  <services>
    <serviceNamespace>xs:anyURI</serviceNamespace> + 
    <recordTypesNamespace>xs:anyURI</recordTypesNamespace> *
  </services>)
</getDataManager>

Impacts of this enhancement

1. Command line client needs to expose the new query functions