Raven Provisioning Service
Spiral 2 Year-end Project Review
Department of Computer Science
University of Arizona
PI: John H. Hartman, Scott Baker
Students: Jude Nelson
August 31, 2010
Sponsored by the National Science Foundation
Project Summary
•
•
Raven is a suite of tools that support long-running GENI experiments:
Stork: secure package management
–
•
IFTD: efficient and fault-tolerant data distribution
–
–
–
•
Enables package management actions on collections of distributed machines
Platform-specific package actions
Raven: experiment deployment tool
–
–
•
Used to distribute packages and Raven metadata
Supports multiple protocols including HTTP, FTP, CoBlitz, BitTorrent
Automatic fail-over
Tempest: distributed package and configuration management
–
–
•
Only installs packages that are signed by a trusted principal
Simple interface for creating and deploying experiment packages
Ties all the other tools together
Owl: experiment monitoring
–
–
Simple client-side scripts collect data, store in centralized MySQL database
Data can be retrieved in HTML, XML, and JSON formats
Sponsored by the National Science Foundation
August 31, 2010
2
Milestone & QSR Status
ID
Milestone
Status
On
Time?
On
Wiki?
GPO
signoff?
S2.a
Support continuous operation of
GENI.
The Stork repository is now continuously
available.
<2
months
late
Yes
Yes
S2.b
GUSH integration
GUSH users can use Raven tools to
install software
Yes
Yes
Yes
S2.c
Plan for using Raven as a
provisioning service for one other
cluster
Plan for porting Raven to ProtoGENI on
wiki.
Yes
Yes
Yes
S2.d
Use publish/subscribe service to
distribute metadata
Publish/subscribe system is installed and
in-use.
Yes
No
No
S2.e
Raven release 3.0
Now using continuous release model
Yes
No
No
S2.f
Better transfer mechanism
IFTD is being tested and deployed.
<2
months
late
No
No
S2.g
Spiral 2 ID management
Compliant with new GID format.
Yes
Yes
No
QSR: 4Q2009
Complete
Yes
Yes
Yes
QSR: 1Q2010
Complete
Yes
Yes
Yes
QSR: 2Q2010
Complete
Yes
Yes
Yes
Sponsored by the National Science Foundation
August 31, 2010
3
Accomplishments 1:
Advancing GENI Spiral 2 Goals
•
•
•
•
Continuous Experimentation: Raven supports long-term experiments.
Tempest and Stork allows packages to be installed and updated during an
experiment, and an experiment’s configuration to be modified.
Instrumentation & Measurement: The Owl monitoring service is designed to
measure and monitor an experiment.
Interoperability: Raven tools have been demonstrated on the PlanetLab,
GpENI, VINI, and Seattle platforms, allowing a single experiment to span
these platforms.
Identity Management: The Raven tools support the uniform GID format. The
Raven team participated in the design of the new format. The Raven tools
also have prototype compatibility with the new XML-based credentials and are
awaiting deployment of those credentials on the control frameworks for further
testing.
Sponsored by the National Science Foundation
August 31, 2010
4
Accomplishments 2:
Other Project Accomplishments
•
•
•
Participation in designing the uniform GID format.
Design and development of Canopus, a calendar-based resource allocation
system to replace Sirius on the PlanetLab cluster.
Creation of a video tutorial for publishing an experiment using Raven tools.
Sponsored by the National Science Foundation
August 31, 2010
5
Issues
•
•
•
Resource specification is a moving target. How do experiments specify the
resources they want and learn the resources they have available across all
platforms?
Lack of a uniform mechanism for ‘initscripts’ hinders rapid deployment of Stork
and Tempest. Within PLC-like environments, initscripts must be manually
installed by the administrator. Other control frameworks (for example,
ProtoGENI) use different mechanisms, requiring a custom means of
bootstrapping Stork and Tempest for each control framework. We suggest the
control frameworks adopt some uniform mechanism for setting non-resource
attributes like initscripts.
Determining user identity (i.e. public keys, GIDs) from within slivers differs
between control frameworks and even between components within control
frameworks, requiring custom code development. We suggest the control
frameworks create a specification for a minimum on-component environment
that includes the component GID and the allowed users’ GIDs.
Sponsored by the National Science Foundation
August 31, 2010
6
Plans
•
Spiral 2
–
–
•
Raven release that includes IFTD
Add authentication and security to Owl
Spiral 3
–
–
–
–
–
–
Better GUIs for Owl and Raven
Pub/sub as a stand-alone service for experiment control
Additional Raven video tutorials
Better cross-platform support, esp. ProtoGENI
Better VINI integration, allowing VNIT topology to be specified via Raven tools
Add time-series support to Owl
Sponsored by the National Science Foundation
August 31, 2010
7