CosmosDataReportingComponent10

Back to Data Reporting Design

Overview

As the architecture of COSMOS becomes more defined tooling is required to facilitate the adoption of the COSMOS platform and components.

One of the main the focus of the data visualization sub project is to provide reusable user interface components to help facilitate the development and validation of Management Data Repositories (MDRs) as defined by the Configuration Management Database Specification Federation specification (CMDBf). An MDR is a CMDBf term that represents a component that contains data about managed resources (e.g. computer systems, application software and building) and/or process artifacts (e.g. incident records and requests for change forms) and the relationships between them. Consider the following high level architecture of the COSMOS platform.

The COSMOS project will not be providing a federating CMDB, but rather the building blocks that can be used in a federated scenario. One of these building blocks, and a key component of the COSMOS platform, is the data broker. The data broker is a middleware component that provides a means to lookup a set of management data repositories (MDRs). Adopters of the COSMOS platform will provide a set of MDRs to access their systems. The data broker may be used by a federating CMDB to lookup the endpoints of the MDRs that have registered with it. Another part of the COSMOS platform is the management domain. The management domain provides the necessary bootstrapping that provides the client with an entry point into the COSMOS runtime. The management domain provides a registry mechanism for the data brokers.

From the perspective of COSMOS, it is only necessary to provide an initial view to visualize the contents of the management domain. Specifically, a user interface is required to show the list of brokers that are registered with the management domain. Consider the following mockup that shows the various views involved in the COSMOS UI. Note each view is highlighted in red.

• The Navigator view will initially show the contents of the management domain in a tree structure. The top level nodes will show a list of brokers registered with the management domain. The second level of the tree will show the list of MDRs registered for each broker.
• The properties view shows a collapsable table that shows properties on a particular tree node. For example, when a user clicks on an MDR the properties view will show the known properties such as the classification, query service the MDR supports and the EPR adddress.
• The details page will show a visualization that knows how to render the contents of the MDR that is selected.

Note: it is expected that the same type of content will be offered by multiple MDRs, for example, more than one management system provides configuration information. As the way content is represented becomes standardized, e.g. SML, the views that render the content will become part of the building blocks offered by COSMOS. Common data representation and the decoupling of views from the MDRs facilitates the rapid customization and specialization of the COSMOS user interface.

The COSMOS User interface is a web interface that is based on a framework that uses Web 2.0 concepts. These concepts are outlined in the data visualization design document [[1]].

Design

At the very basic level the views in COSMOS are constructed from a set of web gadgets building blocks. These gadgets are very simple so that they can be reused to construct different user interface views. For example, the COSMOS user interface provides a gadget that visualizes a tree. An "event wiring point" is defined at each tree node that allows an event to be triggered and published to a communication bus. One possible function of this wiring point is to trigger the repaint event of another widget. Each gadget has a corresponding data feed that is served by a REST service. Consider the following diagram.

Note the current implementation of each REST service provides a JSON data feed. The structure of the JSON data is consumed by a JavaScript class that visualizes the tree. Note that the JSON data structure provides a contract between the backend data and the user interface. This architecture provides a clear decoupling between the raw data and the user interface. As a result, new UI widgets that conform to the JSON contract can be constructed to create different visualizations.

The COSMOS user interface framework provides a set of extension mechanisms:

1. Widgets - A set of widgets can be registered to a specific UI tag. This allows for custom widgets to visualize custom data.
2. View layouts - the layout of the views are defined by an XML file
3. Styles - cosmetic styles are defined in style sheets
4. Data Feeds - data feeds can be associated with a particular widget to produce different visualizations
5. Report templates - reports templates are constructed using the BIRT framework

The following sections define the COSMOS user interface views. Each section is then broken down further to define the gadgets and data feeds needed.

Management Domain Widget

A widget is required to visualize the contents of the data broker. This view will be composed of a tree widget that will show a list of brokers registered with the management domain. Under each broker a list of associated MDRs are shown.

Mockup

There are two forms of the navigator that will allow the user to navigate the list of MDRs that the COSMOS platform is aware of.

The following is the broker view that organizes the MDRs based on brokers.

The following will show the list of MDRs that all the brokers are aware of.

Published Events

• Select - When the user clicks on a node within the tree the widget publishes a 'Select' event to the event bus that contains the node id. This will notify any widget listening to the event bus that a node was clicked with a particular id.

Subscribed Events

• None

Widgets

• Tree View - this widgets consumes a JSON data feed that contains a list of MDRs organized by classification. The widget shows the MDR names in the tree node. When the user clicks on a tree node an event is thrown to the properties widget with the MDR id.
• Properties Table - this widget listens to the event bus for an event that contains a MDR id. The id is then sent to a REST service that provides the meta data for the MDR. An action button is rendered that sends an event with the MDR id to the parent widget to visualize the custom MDR UI.

Data Feeds

• Managed Domain feed - Provides a JSON feed that contains the list of brokers that are registered with the Domain. Note the JSON feed should conform to a tree data structure. The following shows a JSON feed that lists 3 brokers. The first broker contains 3 MDRs while the other two brokers have 1 MDR each.

{ identifier: "object",  label: "title",  
   items:[
       {nodeClass:"broker",title:"Broker 1",object:"9106612", children:[{reference:"13064916"},{reference:"1845558"},reference:"14005505"}]},
       {nodeClass:"broker",title:"Broker 2",object:"9106613", children:[{reference:"55064916"}]},
       {nodeClass:"broker",title:"Broker 3",object:"9106614", children:[{reference:"14043505"}]},
       {nodeClass:"mdr",title:"MDR1",object:"1845558"},
       {nodeClass:"mdr",title:"MDR2",object:"13064916"},
       {nodeClass:"mdr",title:"MDR3",object:"14005505"},
       {nodeClass:"mdr",title:"MDR4",object:"55064916"},
       {nodeClass:"mdr",title:"MDR5",object:"14043505"},
   ]
}

CMDBf Query Builder

This is a generic view that provides a user interface to allow the end user to submit CMDBf queries to an MDR that supports CMDBf queries. This view is composed of a widget that allows the user to construct a CMDBf query and submit the query to a REST service. A results view will render the graph response to the screen.

Mockup

CDMBf Query View

CDMBf Response View

Widgets

• CMDBf Query View - initially this widget will be a simple text box that the user can freely construct a CMDBf query with a submit button or the user can submit a predefined query stored in a file. The submit action will trigger an event which is sent to the CMDBf Query Response widget along with the CMDBf query.
• CMDBf Query Response View - this widget will visualize the graph response within a tree view. The tree will only show the nodes that are part of the CMDBf graph response specification. Any additional information is shown in the details view. Note that a custom visualization for the addiationl information can be created. For example, consider the following graph response returned by a SML repository.

<queryResult xmlns="http://schemas.cmdbf.org/0-9-5/datamodel">
	<nodes templateId="items">
		<item>
			<record	recordId="D:/temp/test-resources/rackUnit1-1.xml">
				
					<s:rackUnit description="The first rack in the first unit" displayName="Rack Unit" name="RackUnit1-1" xmlns:ft="http://www.cosmos.rm/dataCenter/facets" xmlns:it="http://www.cosmos.rm/dataCenter/core" xmlns:pm="http://www.cosmos.rm/dataCenter/properties" xmlns:rf="http://www.cosmos.rm/dataCenter/rackFrame" xmlns:s="http://www.cosmos.rm/dataCenter/rackUnit" xmlns:sml="http://schemas.serviceml.org/sml/2007/02" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.cosmos.rm/dataCenter/rackUnit rackUnit.xsd">
					
						<it:facets>
							<ft:location name="location">
								<ft:location/>
								<it:properties>
									<pm:position coordinate="R10 B-1" name="position">
										<pm:position name="positionChild"/>
									</pm:position>
					
									<it:propertyRef sml:ref="true">
										<sml:uri>
											rack1.xml#xmlns(s=rackFrame)xmlns(it=http://www.cosmos.rm/dataCenter/core)xpointer(/s:rackFrame/it:facets/it:facet[3]/s:rackSockets/it:properties/s:rackSocket[@position="0"])
										</sml:uri>
									</it:propertyRef>
					
								</it:properties>
							</ft:location>
							<ft:assetManagement name="assetManagement">
								<it:properties>
									<pm:serialNum identifier="X4792" name="serialNumber"/>
								</it:properties>
							</ft:assetManagement>
							<s:rackUnitConfig name="configuration">
								<it:properties>
									<pm:processor name="processor">
										<pm:vendor name=" Intel "/>
										<instructionSet label="x86"/>
										<clock multiplier="G" speed="2"/>
									</pm:processor>
									<pm:ram name="ram">
										<pm:size amount="2000" multiplier="M"/>
									</pm:ram>
									<pm:disk name="disk">
										<pm:size amount="120" multiplier="G"/>
									</pm:disk>
									<pm:networkPorts name="ports">
										<pm:port number="0">
											<pm:IPAddress name="address" value="10.23.78.234"/>
										</pm:port>
										<pm:port number="1">
											<pm:IPAddress name="address" value="10.23.79.232"/>
										</pm:port>
									</pm:networkPorts>
								</it:properties>
							</s:rackUnitConfig>
							<ft:location name="location">
								<it:itemRef sml:ref="true">
									<sml:uri>
										Cable1.xml
									</sml:uri>
								</it:itemRef>
							</ft:location>
							<it:facetRef name="reference" sml:ref="true">
								<sml:uri>
									Cable1.xml#xmlns(s=cable)xmlns(it=http://www.cosmos.rm/dataCenter/core)xpointer(/s:cable/it:facets/it:facet[1])
								</sml:uri>
							</it:facetRef>
						</it:facets>
					
					</s:rackUnit>
			</record>
		</item>
	</nodes>
	<edges templateId="reference">
		<relationship>
			<sourceItem>
				<mdrId>http://cosmos.org/rm/sml/repository</mdrId>
				<localId>D:/temp/test-resources/rackUnit1-1.xml</localId>
			</sourceItem>
			<targetItem>
				<mdrId>http://cosmos.org/rm/sml/repository</mdrId>
				<localId>D:/temp/test-resources/Cable1.xml</localId>
			</targetItem>
		</relationship>
	</edges>
</queryResult>

The XML fragment that is encased in the "queryResult/nodes/item/record" node has a specific structure pertaining to the SML Repository that was queried. This structure conforms to the SML specification. The COSMOS UI will provide a default view to visualize XML structured data. This is explained in the XML Viewer section. It is also possible to create a custom visualization to visualize the SML structure.

Published Events

CDMBf Query View

• Select - When the user clicks on a node within the tree the widget publishes a 'Select' event to the event bus that contains the node id. This will notify any widget listening to the event bus that a node was clicked with a particular id.

Subscribed Events

CDMBf Query View

• Select - The query view will subscribe to the event bus that publish the 'Select' event when the user clicks on a MDR node within the Broker View.

CDMBf Response View

• Select - The query view will subscribe to the event bus that publish the 'Select' event when the user submits a CMDBf query from the CMDBf Query View.

Data Feeds

• CDMBf Response View - this data feed takes in a CMDBf query and returns a JSON feed that represents an XML graph response.

XML Viewer

This is a view that understands how to visualize the content returned by an XML fragment. This view is composed of a tree view and a properties view.

Mockup

Widgets

• Tree View - visualizes a set of nodes that is specified by some metadata. When the user clicks on a tree node an event is thrown to the properties widget with the resource id.
• Properties Table - this widget listens to the event bus for an event that contains a resource id. The id is then sent to a REST service that provides the properties for the XML node selected.

Published Events

Tree View

• Select - The tree view will publish a 'Select' event containing the id of the node..

Subscribed Events

Properties Table

• Select - The properties table will subscribe to the event bus that publish the 'Select' event when the user clicks on a node in the tree view.

Data Feeds

These feeds will use CMDBf queries to query the MDR and return a graph response. A JSON feed will be constructued from the graph response.

• XML fragment - provides a JSON feed that contains the content of the XML fragment.
• Resource properties - provides a JSON feed that contains a list of properties for a particular XML node. Note the JSON feed should conform to a table data structure.

It should be noted that this view can be used to visualize any form of XML content. The feeds act as a filter to determine what gets visualized in the Tree and Properties view. Consider the following diagram that shows how the web ui is separated from the data.

Note that the ui widgets consume JSON data structures that are produced by a set of servlets. An "outputter" class is associated with each servlet. The purpose of the outputter class is to take in some form on input object and transform the contents of the output to JSON. The outputter class contains specific transform domain logic. As a result the outputter class is aware of the input format and the output format that a specific widget requires. This architecture will allow the support of additional output formats other than JSON. Furthermore if the input data structure changes the changes are isolated in the outputter class.

Now consider extending this framework to provide a generic JSON outputter that converts an XML fragment to the JSON data structure that the Tree widget requires. The following is a sample JSON representation of the Tree data.

{ identifier: "object",  label: "title",  items:[
   {nodeClass:"datasource",title:"datasource - tomcat 23 server",object:"1", children:[{reference:"5"}]}
   {nodeClass:"datasource",title:"datasource - IBM websphere 14 server",object:"2"},
   {nodeClass:"datasource",title:"datasource - JBoss 681 server",object:"3"},
   {nodeClass:"datasource",title:"datasource - tomcat 6 server ",object:"4"}
   {nodeClass:"childNode",title:"tomcat child node",object:"5"}
]}  

Note that each tree node has 3 to 4 pieces of data:

1. nodeClass - a tag that adds cosmetic decorators to the node
2. title - the text to display
3. object - unique id to identify a node in the tree
4. children - list of children node ids

Every XML node in the XML fragment are converted to this basic representation. The relationship between the XML nodes are specified as a list of children ids. A XSL stylesheet can be created to convert the XML fragment to the JSON format specified above. A generic outputter class can be created to transform the XML fragment to JSON.

Note that the XSLT files are stored in a predefined configuration directory on the server. As a result the client can make REST calls to use a specific XSLT file. Here are some sample REST calls:

http://localhost:8080/COSMOSUI/NavigatorJSON?xslt=SML2JSON
http://localhost:8080/COSMOSUI/NavigatorJSON?xslt=CBE2JSON
http://localhost:8080/COSMOSUI/NavigatorJSON?xslt=Stat2JSON

On the server the XSLT files are deployed in the following directory respectively.

C:\apache-tomcat-5.5.20\webapps\COSMOSUI\views\cosmos\XML2JSON\SML2JSON\xml2json.xslt
C:\apache-tomcat-5.5.20\webapps\COSMOSUI\views\cosmos\XML2JSON\CBE2JSON\cbe2json.xslt
C:\apache-tomcat-5.5.20\webapps\COSMOSUI\views\cosmos\XML2JSON\Stat2JSON\stat2json.xslt

In future iterations this transform mechanism should be revisited to find an easier way for adopters to create the transform logic.

Statistical/CBE View

This is a custom view that will present the user with a list of data sources and a properties view. For example, consider an MDR that contains statistical data for several tomcat servers and operating systems. This view will list all the set of tomcat servers and operating servers as data sources contain statistical data. The user has the option to click on a particular data source. At this point a properties table displays the meta data information for the data source (e.g. data source name, type of data collected, etc.). In addition a drop down menu is shown on the data source node to generate a report based on a set of available statistical reports.

Mockup

Widgets

• List View - visualizes a set of data sources. The widget shows the data source names as list nodes. When the user clicks on a node an event is thrown to the properties widget with the data source id.
• Properties Table - this widget listens to the event bus for an event that contains a data source id. The id is then sent to a REST service that provides the properties for the data source.
• Available Report Pop-up Menu - a popup menu button that shows the available reports that can be used to visualize the data. When the user selects a particular report an event is sent to the BIRT report viewer widget.
• BIRT report viewer - This sends a request to the BIRT reporting system to render the report given the data source id and report template id.

Data Feeds

These feeds will use CMDBf to query the SML repository and construct the JSON feed.

• SML resource list - provides a JSON feed that contains the list of resources. Note the JSON feed should conform to a tree data structure.
• Resource properties - provides a JSON feed that contains a list of properties for a particular resource. Note the JSON feed should conform to a table data structure.
• Available report list - provides a JSON feed that contains a list of report template names and ids that can be used for a particular data type (i.e. statistical, CBE).

Broker View Overview

PERT Chart

1. Refactor tree widget to run with Dojo 0.9 release
2. Create Broker tree widget
3. Create data feed that contains management domain and MDR information
4. Add Junits for Broker tree data feed
5. Add manual tests to test broker widget
6. Create Simple table properties widget
7. Create data feed that contains properties of Broker or MDR
8. Add Junits for properties data feed
9. Add manual tests to test properties widget
10. Create CMDBf query widget
11. Create data feed that takes a CDMBf query widget and returns a graph response
12. Add Junits for CMDBf query data feed
13. Add manual tests to test CMDBf query widget
14. Create CMDBf response view
15. Configure properties widget to show CDMBf response properties
16. Add manual tests to test CMDBf response widget
17. Add manual tests to test CMDBf properties widget
18. Integrate CMDBf response view with Broker tree widget
19. Construct XML tree widget
20. Configure properties widget to show XML tree widget
21. Construct data feed to visualize XML fragments
22. Add manual tests to test XML tree widget
23. Add manual tests to test XML properties widget
24. Add Junits for XML fragment data feed
25. Construct statistical MDR list tree widget
26. Create data feed that contains the list of data sources the statistical MDR is aware of
27. Create data feed that contains the list of data sources the CBE MDR is aware of
28. Construct pop-up menu action that list the available reports on a particular datasource
29. Configure properties widget to show statistical/CBE datasource properties
30. Add manual tests to test pop-up menu widget
31. Add manual tests to test datasource properties widget
32. Add manual tests to test statistical tree widget
33. Add Junit for data source data feed
34. Integrate views into a central user interface
35. Add manual tests for integrated view

Work Item Break Down

person	work item
person1(Broker View)	1,2,3,4,5,6,7,8,9
person2(CMDBf Query View)	10,11,12,13
person3(CMDBf Response View)	14,15,16,17,18
person4(XML View)	19,20,21,22,23,24
person5(Statistical/CBE View)	25,26,27,29,31,32,33
person6(Report Widget)	28,30
person7(UI Integration)	34,35

Glossary

• JSON (JavaScript Object Notation) [2] is a light-weight interchange format. It is used as the interchange format because the Javascript widgets can easily serialize and deserialize Java objects over the network.