COSMOS CMDBf Framework XSLT Redesign

This is the design for bugzilla 242137.

Change History

Name:	Date:	Revised Sections:
Ray Ellis	June 24, 2008	Initial version
Ray Ellis	June 25, 2008	Example application added
Ray Ellis	Aug 6, 2008	Query service prototype added

Proposal

Overview

While looking at some performance and robustness concerns that came up when applying the current CMDBf framework, it occurred to me that one way of looking at what we're trying to do is to recognize some patterns in an XML input stream, invoke an application-specific function upon recognizing a pattern, and then format the function responses back into XML. Looked at in this light, an approach based on XSLT has much to recommend it. Recognizing patterns in XML input and generating XML output is precisely what it is intended to accomplish, and implementations are generally efficient and robust at doing so. By dealing with the XML directly you deal with a standards-based artifact, not a derivative, and a whole layer of code and a whole class of potential difficulties can be avoided. Moreover, XSLT has been a w3c standard since 1999. There are numerous commercial and open-source implementations compatible with a range of programming environments, and there are a variety of development tools as well.

Registration Handling

To illustrate the proposed approach, consider the following templates, to process registerRequest messages:

   <xsl:template match="cmdbf:registerRequest">
       <cmdbf:registerResponse>
           <xsl:for-each select="cmdbf:itemList/cmdbf:item">
               <xsl:variable name="itemId" select="InstanceId:new(cmdbf:instanceId/cmdbf:mdrId, cmdbf:instanceId/cmdbf:localId)"/>
               <xsl:variable name="errMsg" select="CMDBfRegistrationHandler:registerItem(cmdbf:record, $itemId)"/>
               <xsl:call-template name="responseTemplate">
                   <xsl:with-param name="mdrId" select="cmdbf:instanceId/cmdbf:mdrId"/>
                   <xsl:with-param name="localId" select="cmdbf:instanceId/cmdbf:localId"/>
                   <xsl:with-param name="errMsg" select="$errMsg"/>
               </xsl:call-template>
           </xsl:for-each>
           <xsl:for-each select="cmdbf:relationshipList/cmdbf:relationship">
               <xsl:variable name="sourceId" select="InstanceId:new(cmdbf:source/cmdbf:mdrId, cmdbf:source/cmdbf:localId)"/>
               <xsl:variable name="targetId" select="InstanceId:new(cmdbf:target/cmdbf:mdrId, cmdbf:target/cmdbf:localId)"/>
               <xsl:variable name="relationshipId" select="InstanceId:new(cmdbf:instanceId/cmdbf:mdrId, cmdbf:instanceId/cmdbf:localId)"/>
               <xsl:variable name="errMsg" select="CMDBfRegistrationHandler:registerRelationship(cmdbf:record, $sourceId, $targetId, $relationshipId)"/>
               <xsl:call-template name="responseTemplate">
                   <xsl:with-param name="mdrId" select="cmdbf:instanceId/cmdbf:mdrId"/>
                   <xsl:with-param name="localId" select="cmdbf:instanceId/cmdbf:localId"/>
                   <xsl:with-param name="errMsg" select="$errMsg"/>
               </xsl:call-template>
           </xsl:for-each>
       </cmdbf:registerResponse>
   </xsl:template>

   <xsl:template name="responseTemplate">
       <xsl:param name="mdrId"/>
       <xsl:param name="localId"/>
       <xsl:param name="errMsg"/>
       <cmdbf:instanceResponse>
           <cmdbf:instanceId>
               <cmdbf:mdrId><xsl:value-of select="$mdrId"/></cmdbf:mdrId>
               <cmdbf:localId><xsl:value-of select="$localId"/></cmdbf:localId>
           </cmdbf:instanceId>
           <xsl:choose>
               <xsl:when test="$errMsg = ">
                   <cmdbf:accepted/>
               </xsl:when>
               <xsl:otherwise>
                   <cmdbf:declined><xsl:value-of select="$errMsg"/></cmdbf:declined>
               </xsl:otherwise>
           </xsl:choose>
       </cmdbf:instanceResponse>
   </xsl:template>

The first template is triggered by a registerRequest message. It loops through each item and relationship in the message, passing information to the extension functions registerItem and registerRelationship, and then formats each reply into an instanceResponse in responseTemplate. Running this transform parses the incoming message, recognizes the item and relationship registration patterns, invokes application-specific functions to process the requests, and formats the response -- without any further code than that required to invoke the transform and implement the application-specific functions. The transform can be invoked from a SOAP request in a JAX-WS environment with just a few dozen lines of code, or it can be driven directly from something like a servlet input stream. I've run standalone tests using a file for the message source, and with the open-source Apache Xalan processor this approach processes large (1-100MB) input messages 4-5 times faster than the current COSMOS framework, with similar memory requirements.

Query Handling

The query interface is more complex, but I think the gains to be made are greater, too. I suspect that most repositories are going to be based on a relational database, generally accessed indirectly through application logic. Even for a well-engineered application, reducing the number of trips you take through the stack to satisfy a CMDBf query can offer substantial performance benefits. I would propose a similar structure to the registerRequest approach outlined above: use XSLT to break down the CMDBf query message, and invoke application-specific functions to synthesize a query from the constraints contained in the templates. The queries would then be evaluated to produce the XML response. XSL has import/include features that I would envision being used to format the application-specific record as well as the standard-specified portion of the message. This was my initial reason to look at the details of the framework, by the way, and I have measured 10X and greater improvement on complex queries using this general approach.

Example Application

I've posted a simple application here that uses the above approach to implement a CMDBf-compliant registration service. It doesn't do much besides accepting all register and deregister requests, but it's useful as a test server and it serves as an example. Here is an prototype of the proposed query service framework, too. It doesn't have quite all the required functions (it doesn't comprehend content selectors, for example), but it does allow query optimization. I have a working application that shows much improved performance using this approach, too, so it has seen some small use.

Detailed Design

... to be written ... --