Ganymede Provisioning Workshop Breakout Results

Day One Results

Non-eclipse pieces

The provisioning mechanism needs to be able to handle and be aware of non-eclipse pieces (see list below). It needs to be able to reason about those non-eclipse pieces even though it has not installed them (e.g. check their presence, get their path, etc.). It needs to be able to install those non-eclipse pieces, which means that the metadata needs to be able to handle those requirements. In addition to that, the provisioning mechanism should support the execution of non-java code while provisioning (e.g. run an ismp installer)

All operations against eclipse and non-eclipse pieces should be structured to help an installation author to keep things consistent, therefore limiting the cases where the system is put in an inconsistent state.

It is clear that the metadata needs to be extensible to describe how to install/configure/discover non-eclipse pieces. Eclipse pieces should be able to express dependencies onto non-eclipse pieces, and get configuration information about them. In the current equinox provisioning design, installing/configuring another runtime can be done by a touchpoint.

Non-eclipse pieces target a "runtime".

Like OSGi/eclipse is seen as a "Runtime" that needs to be provisioned, the "Runtime" analogy can be carried on to talk about the non-eclipse pieces. For example, an operating system can be seen as a Runtime in which dlls are provisioned. The same goes of WAR files for a web app server, or .net asseblies.

Other related points

- On some systems installing non-eclipse pieces requires access rights - We need to be able to know whether the system needs to be restarted

Non-eclipse pieces of interest (non-exhaustive)

eclipse launcher, JREs, Dlls, SOs, Registry entries, Script files, Java jars, System env variable, Browser, MSI, ISMP, WARs

Co-existence of multiple provisioners

• multiple sources
  • consistent UI / operations story
• multiple users
  • Linux experience but Vista issues, too
• metadata compatibility - common basic metadata (install registry) plus source specific properties/extensions
  • how to avoid duplicate information/properties?
• interaction of multiple sources (priority, precedence, preference)
• validation of profiles
• licensing issues
• shared runtimes
• shared binaries
• repository discovery (RSS? Web servcies?)
• disconnected cases
  • media (including metadata and/or artifacts)
  • pending connection - future installation
• on-the-fly creation of native packages (in the background)
  • add dependencies on native bits (ex. adding a dependency on GTK for SWT so that if GTK is going to be removed, SWT will either also be removed or at least the potential for the removal will be presented)
• admin interaction
• touchpoints for native installation methods (ex. MSI, RPM)

Spectrum of Governance : Local control with central governance

• initial provisioning
• ongoing provisioning
  • points of control
    • "I am the governance provider" -- ensure that at the OSGi layer we can control who can activate bundles
  • potential for ongoing communication between provisioning server and client
    • license revocation or mandatory changes for other reasons
• hooks for governance both server and client side
  • validation at provisioning server
    • based on not only resolver but also other policies including software configuration settings
• spectrum of governance
  • fully managed
    • no or almost no client-side ui
    • user has no choice over what is installed and can not install additional software
  • partially managed
    • roles of user can differ (can they choose to install software)
    • governance may be required while end user is installing software
    • governance may be restricted to a certain set of "critical" components
    • potential for some user interface but may be restricted
  • no management
    • complete provisioning ui (comparable to existing update manager UI)

metadata for user

• end user and admins are users
• what is good for me to consume? Ready for prime time?
• Governance:
  • What does this person/role have
  • what is this person/roll allowed to consume? (permissions/access controls)
  • what can people change in the profiles
  • license management ( what licenses have been agreed to, which must be agreed to, license server locations, ...)
  • what profiles can I manage

• what does XX depend on?
• consistency checking (can I add XX)

• discovery
  • what do my friends have?
  • what function is available
  • search for things that fulfill a need
  • categorization of function
  • What is recommended. Ranking? interests? what did other people like like?
  • what repositories are available?
  • something like Mugshot Application usage http://mugshot.org/applications ?

• has this lineup been tested?
  • are users happy/confident
  • known good / known bad
• what NL languages are available

• what are the system requirements?
  • can this run on platform X
  • cross platform validation/qualification
  • machine capabilities (cores, memory, cpu speed, connectivity, ....)
  • download size
• install branding
• Reporting
  • who (how many people) are using a particular lineup
  • what other configurations do I have
  • what other configurations are running
• how do I report bugs, related user forums
• settings and preferences
  • shared by role, organization, user, templates, ...
• install history
• can I roll back
• does this require a system restart or reboot
• what is needed so that I can run disconnected (whatever)
• what are the external constraints for this system (client/server version, ...)
• where do you enter the system?
  • product id? default perspective?
  • Enabled Capabilities, Welcome info?
• source used to build this IU
• extensible attributes of the metadata
• are there any post actions required (e.g., populate workspace, ...)

metadata for system

Resource/runtime/service requirements Platform information

state of installable units What is installed?

Qualification What touch points do you need

Exclusivity Singleton

transports available Internet connection? Speed? Local connection vs end to end What transports available to get this artifact? What repositories contain this metadata/artifact?

What metadata is new since xyzzy?

Who provides beer when the developers make a mistake

What are install operations that will be done?

What installable units will be installed/replaced/uninstalled?

Repository questions - binary deltas, latency, fastest mirrors

Security - is there a proxy

Authorization level Restart required

Will configuration be valid

time/size(download, memory) to perform

licensing consequences

system locale languages to install

languages supported what are your dependencies power consumption

system view/all users view/private view

vendor signed hashes/md5 downloaded correctly

alignment

Suggests/recommends

Does xyzzy support emacs shortcuts?

Day Two Morning Results

Project Intersections

These rough notes were taken during the workshop. The project intersections have since been split into a separate Provisioning Project Intersections page.

(italics means interested party, but not necessarily an active participant)

Metadata

• Everyone

Admin/Advanced UI

• Maya
• Equinox

Central Manager

• Maya
• Installation manager
• Expeditor
• Smartup

Tools

• Maya
• Equinox
• EPP
• Expeditor
• Installation Manager

Director

• Maya
• Equinox
• Smartup
• Buckminster

Engine

• Equinox
• Maya
• Smartup
• Buckminster

End-user UI

• Equinox
• Maya
• Installation Manager

Touch Points

• Equinox
• Maya

Native Installer/Bootstrap

• EPP
• Maya

Repository

• Buckminster
• Equinox
• Maya
• Smartup
• Spaces

Transport

• Equinox
• Smartup
• Maya

Delta Server/mechanism

• Smartup

Profile Management

• Equinox
• Maya
• Buckminster
• EPP

Integration with other mechanisms

• Equinox
• Linux Distros

Cross cutting concerns

• Integration with other mechanisms
• Multiple directors

Core metadata

• Two aspects of metadata: system-level data, that needs to be processed for installing something, and user-level data, necessary for showing meaningful information to the user (e.g. groups of install units, etc.)
• Metadata has to handle both system-level and user-level concerns.
• Metadata has to be extensible.
• Core Metadata: the system-level metadata providing the information necessary for putting the system into a target state
• A grouping mechanism is needed, but cannot be mandatory for installing things. It should avoid duplicating dependency information.
• Allow to express cross-runtime dependencies and requirements (like needing a certain web server, dll, compiler, etc.)
• Cross-runtime dependency: a dependency between two different kinds of runtimes. Example: a bundle in an eclipse-runtime needs a certain web server
• Concepts:
  • "Installable Units" - something that gets installed, contains dependencies and has a descriptor of what the payload is
  • "Grouping" - groups a number of installable units. Groups are not mandatory.
  • "Constraint modification" - allows to express limitations or expand limitations. Should allow to declaratively overwrite and change metadata to fix "screw-ups".

User workflows

• lots of different workflows for products
• we like the Firefox simple and low-key model
• Maya workflow:
  • user:
    • 1. start bootstrap
    • 2. pick profile from most popular
    • OR customize profile with list of available stuff
    • 3. launch profile
  • admin:
    • create / modify profiles
    • distribute profiles to others
• New user workflow:
  • potential for many configuration screens (ex. location, licence, information collection, password)
  • progress dialogue during download/installation (same for media-based installation)
• Update workflow:
  • ideally there will be no need to "start over" with complete new zip
  • do away with infinitely growing footprint
  • look into keeping old / keeping n versions
    • require API or per-product policy
    • being referred to as "garbage colletion policy"
  • compared with today, many fewer choices presented to user
  • potential preferences:

    • [ ] Automatically download

      (on by default ideally)

    • [ ] Automatically install

      (not on by default)
• Areas of concern:
  • updating of non-user-writable bundles
    • interaction between Eclipse-based updater and system-wide updater
    • policy surrounding updating shared installs
    • support some sort of admin proxy (ex. PAM dialogue, admin password prompt)
  • profiles shared between machines
    • workspace, metadata (describing "profile"), user site (optional)
    • take some combination of above to a new machine and replicate system
      • perhaps some Import/Export mechanism

Day Two Afternoon Results

Engine/Director deep dive

Context on IBM install manager terms:

• A "Job" is either install, update, rollback, fix, uninstall
• A list of jobs is provided to the director
• Multiple jobs can be part of a single interaction with the director

Common work flow:

• Some piece of software has been installed
• User initiates an install/update interaction
• Question to director: what is installed right now?
• Question to discoverer: What other things are available to be installed?
• Question to discoverer: What is this user allowed to install?
• Show user what is installed right now and what can be installed
• User selects pieces of software to install/uninstall/replace
• Question to director: What are the prerequisites for things to be installed? Are the selected actions valid?
• Question to director: What are the resource implications of performing this set of jobs? (disk space usage, etc)
• Question to engine: How much work is it to do these actions?
• Answer comes back from director, and consequences are displayed to user
• User says ok and asks for the work to start
• Question to director: Please install/uninstall/update the following things...
• Question to engine: Please install/uninstall/update the following things...
• Engine goes through install phases:
  • Fetch/Collect.
    • Ask touchpoint what needs to be done for each artifact
    • Touchpoint may return a request to be performed (e.g., mirror request)
    • Download manager is told to perform the mirror request
  • Install
  • Configure

Common Maya workflow:

• Startup Maya
• Question to director: What profiles are currently installed?
• Question to director: What profiles are available to be installed?
• User selects/creates a profile
• Question to ???: Please launch this profile

Multiple directors:

• Governance settings may dictate that some things cannot be modified, but other things can be installed/uninstalled at the user's perogative.
• Can we have one director for a subset of the system, and another director for other parts?

Breaking down the Director

As we fleshed out the concrete role of the director, we found it useful to break down the director into three different areas:

Director:

• Contains the main API to query/perform provisioning
• Well-behaved director consults Governor for permission to do things
• Director consults some "Discovery" service to find other parts of the provisioning system

Governor:

• License issues
• User access control
• Security model
• Governor may be local or remote
• Governor is optional - in simple cases you might not have one
• Governor is a policy integration point

Discovery:

• How to find governor
• How to find director
• How to find install registry

Action opportunities

• Create provisioning-dev mailing list. Announce availability of email list and wiki pages. -- Jeff due 4/11

• Manage mailing list conversation by announcing specific topics every few days and folding conversation into refactored wiki pages. -- Sharanya, Markus, Elias (ongoing every few days)

• Metadata catalog, or what ever turns out to be appropriate. -- Dave, Filip 4/27

• Update system diagram and develop glossary of terms (add to GPW Lexicon). -- Tim, Jeff 4/20

• Flesh out 'project intersections' Consider annotating each item with full sentences. Consult the community. -- John, Filip 4/20 (gated by system diagram) Update: page created: Provisioning Project Intersections, added description of current Buckminster solution

• Develop an initial list of potential work items based on project intersections identified above and cross-cutting concerns. These will include specific contribution intentions. Prioritize these in some open way. These will be eventually cast a bugs and assigned to people. -- Tim, Jeff 4/27

• Collect requirements for multi-user installs -- James, Andrew 4/20 Update: see Multi-User_Install_Concerns

• Draft more detailed requirements of Governor and Director (decomposition). Possibly include exploratory implementations. -- Jeff, Tim, Pascal, James 4/25

• Encourage cross-committers within the space of provisioning. Count the number of such as a measure of cooperation between projects. Relate this to the new concept of mentors. -- Ward 4/20

• Draft a timeline that at least shows various decision timeframes and commit dates. Index each project's roadmaps as they are developed and adjusted. Indicate the degree of confidence for each item. -- Sharanya, Pascal 4/20

• Develop bugzilla query for all provisioning related bugs for all projects. The goal is to enable tracking of detailed development across projects. Post this query on this wiki page. Update it when provisioning projects change. -- Elias 4/20 Update: see Bugzilla Query (Provisioning)

• Develop technical plan for alignment of Maya and Equinox. -- Dennis, Pascal 4/27