MaturityAssessmentHudson

The whole Maturity Assessment process has been automated and is executed on a Hudson instance setup on the dashboard.polarsys.org server. The Hudson instance can be accessed here: http://dashboard.polarsys.org/hudson.

Access is required for any modification. If you would like (and have good reasons) to access the instance, please drop us an email and we will create an account for you.

Structure of Hudson jobs

Each project analysis process is decomposed in three jobs for a better modularity and reuse of steps. These steps are (in order of execution):

• <project>_run_analysis is the main launcher. It is based on the template-run-analysis project, which:
  • executes the rule_checking job,
  • get the generated artefacts from the rule_checking job (XML results from rule-checking tools),
  • executes the retrieve_data job,
  • gets the artefacts from the retrieve_data job (json files for all data sources),
  • executes the template-publish-site job to regenerate the dashboard web site with the new data.
• <project>_rule_checking manages rule-checking tools. It is based on the template-rule-checking project, which:
  • retrieves the source,
  • analyses the source with PMD,
  • retrieves the jars from an hudson instance,
  • analyses the jars with FindBugs.
• <project>_retrieve_data really computes and retrieves metrics. It is based on the template-retrieve-data project, which: :
  • retrieves PMI information,
  • retrieves Marketplace information (note that this data is not used any more in the current process),
  • retrieves the Grimoire files (for SCM, MLS and ITS),
  • retrieves SonarQube data, and
  • executes the analyse_rules script to extract needed metrics from rule-checking tools XML results.

The process makes an intensive use of template projects. This plugin enables us to create inactive projects containing all of the business logic in their build steps, which we can reuse in the real project jobs. That way, if it is needed to change something in the analysis process, we only need to modifiy the (inactive) template project and all further executions of real projects will use the updated definition. Besides better maintainability, it ensures that all projects use the exact same series of steps.

Scheduling

Projects are analysed every day at 0:30 (EDT). Their execution is organised so only a few of them are run at the same time. A RSS feed can be fetched from Hudson, either from the run-all project (using http://dashboard.polarsys.org/hudson/job/run-all/rssAll ) or from each project individually (using http://dashboard.polarsys.org/hudson/job/project-run-analysis/rssAll ).

All files matching the /projects/*/*_comments.json pattern are committed in the git repository every day at 0:00 (EDT), as a backup of the projects' comments.

All files matching the WEB/projects/*_attributes.json, WEB/projects/*_indicators.json, and WEB/projects/*_metrics.json patterns are committed in the git repository every day at 0:00 (EDT), as a backup of the projects' data.

[WIP -- TBD] All files matching the WEB/projects/*_attributes.json, WEB/projects/*_indicators.json, and WEB/projects/*_metrics.json patterns are copied to WEB/projects/*_[type]_hist.json for the display of trends once a week (on tuesday) at 6:00 (EDT), after the projects' analysis.

The Thinbackup plugin is used on the instance. Full backups are executed every sunday at 12:00 (EDT) and differential backups are executed everyday (excepted Sunday) at 12:00. All data files are saved in the /home/boris/hudson_backups/ directory.

Adding a project to the process

First make sure you have all required information, as specified in the MaturityAssessmentCheckList.

Creating a new project in the Hudson instance is done as follows:

• Create a new project named myproject-rule-checking. Select Copy existing job in the creation panel, and select e.g. capella-rule-checking (or whatever *-rule-checking project) as origin.
  • The parameters will be provided during run time by the calling project, you can leave them empty.
  • In the Source Code Management section, change the Git URL to the project's repository.
  • That's all.
• Create a new project named myproject-retrieve-data. Select Copy existing job in the creation panel, and select e.g. capella-retrieve-data (or whatever *-retrieve-data project) as origin.
  • That's all, since values will be inherited (passed on) during execution.
• Create a new project named myproject-run-analysis. Select Copy existing job in the creation panel, and select e.g. capella-run-analysis (or whatever *-run-analysis project) as origin.
  • Change the ID (the ID is the project's name in the Eclipse/Polarsys infra, e.g. tools.cdt or polarsys.capella).
  • Changing the URL_MKT is optional, since metrics from the marketplace are not used as of today.
  • Change the URL_ZIP to a link to the zipped update site on the project's Hudson instance. As an example for Capella it is: https://hudson.polarsys.org/capella/job/capella-master/lastSuccessfulBuild/artifact/result/publish/org.polarsys.capella/site.p2/plugins/*zip*/plugins.zip
  • In the build steps, replace the name of the projects to be launched by <project>-run-analysis (three times) or <project>-retrieve-data (twice)
  • That's all.
• Open the template-publish-site job, and add a Copy artifacts from another project build step.
  • Select <project>-run-analysis as project name,
  • select Latests successful build,
  • specify dist/* as artefacts to copy, and
  • projects/<project_id> as target directory (e.g. projects/polarsys.capella).

Then execute the <project>-run-analysis project, and check that all metrics are retrieved correctly and published.

Other projects

There are a few other projects defined in the Hudson instance:

• template-publish-site is never instantiated, and simply generates the static part of the dashboard (i.e. everything excepted commenting system). It merely gathers metrics from all projects and executes the publish_static perl script from the PolarSys repo. If the build is successful, it copies the generated files to the htdocs directory of the server.
• run-all simply executes all projects, either sequentially or in parallel with a smart sequence to not overload the server. This is the job to be run for the whole analysis, and the one to be executed daily to refresh the dashboard. Note that if you change the default settings of projects (ID, URL_ZIP) and want to use the run-all project, then you must also change the corresponding parameters used to execute the jobs in the run-all project.