ConfValley: A Systematic Configuration Validation Framework
for Cloud Services
Peng
†
Huang ,
‡
Bolosky ,
William J.
Abhishek Singh*, and Yuanyuan
†
‡
UCSD , Microsoft Research , Microsoft*
Background
Problem & Motivation
Configuration in cloud-scale systems
Configuration validation
 Wide variety
o Format: XML, INI, YAML, REST API, etc.
o Control: feature, security, fault tolerance, performance, etc.




 Interwoven impact
o Config. in one component can impact others
 Duplication & customization
†
Zhou
Specs in CPL
 Quick comparison
What: checking if configuration satisfies some constraints
When: configuration edit-time, integration, production
Benefits: prevent damages from production, save diagnosis
Current practice: inefficient, ad-hoc, and late
o Manual configuration review
o Bulky ad-hoc scripts/code in different places
o Invoked late in runtime
specs in CPL
ad-hoc config. validation code
 Example CPL specs
conditional specs
set configuration sources, constraint macros
one-liner spec
advanced specs with data piping
Misconfiguration is expensive
* data from a major cloud provider
Lifecycle of (mis)configuration
ad-hoc config. validation code (a)
ad-hoc config. validation code (b)
ad-hoc config. validation code (c)
control validation scope
Solution
Evaluation
ConfValley: a config. validation framework
 Rewrite existing validation code




Language: a declarative language (CPL) to write config. specs
Abstraction: unified config. representation
Inference: infer basic specs automatically
Validation: support different scenarios,
o Edit-time, in IDE
o On the fly, an interactive console
o Continuous service, run specs when config. changes
Config.
Type A
Type B
Type C
Orig. code
LOC
800+
3300+
180+
LOC
50
109
14
Specs in CPL
Count
Inferable
17
6
62
27
6
1
Dev. Time*
Man-hour
1
6
0.5
Rewrite validation code for three kinds of configuration data used inside Microsoft Azure into
specs in CPL. *: time it takes to understand the original code and then write specs in CPL.
 Prevent misconfiguration
Config. branch
Trunch
Branch A
Branch B
Reported errors
4
2
2
Run expert-written CPL specs on three latest configuration branches inside Microsoft Azure
 Performance
Config. Instances
Type A
Type B
Type C
44,102
1,969,588
1,529
Specs in CPL
Count
Source
Seq.
182 Inferred, optimized 10
62
Human-written
518
95
Inferred
0.4
Time
P10.Min P10.Med. P10.Max
2
2
4
49
52
208
0.3
0.3
0.3
Run CPL specs on three kinds of configuration data used inside Microsoft Azure