A survey of submodular
functions maximization
Yao Zhang
03/19/2015
Example
• Deploy sensors in the water distribution
network to detect contamination
• F(S): the performance of the detection
when a set S of places is selected
2
Definition of submodular functions
• finite ground set V={1,2,...n}
• set function f(S):
• Marginal gain:
• Submodular: diminishing return
3
Definition of submodular functions
• Modular function:
• Supermodular
4
Examples of submodular function
• Deploy sensors in the water distribution network to
detect contamination
From Krause’s survey
5
Examples of submodular function
• From Krause’s survey
•
•
•
•
•
Weighted coverage functions
Entropy
Mutual information
Cut capacity functions
…
• Influence function (Kempe 03)
• f(S): the expected number of infected nodes when
nodes in S are infected at the start
• Propagation model
• Linear threshold
• Independent Cascade
6
Properties of submodular function
• Linear combination
• if set functions F1,...,Fm are submodular
functions, and a1,...,am>0
• then
is submodular
• Concavity
7
Submodularity optimization
• Except for the submodularity, we assume:
• 1. Monotonically non-decreasing
• 2.
8
Summary of submodularity optimization
For more details, see
http://submodularity.org/
9
Focus on this part
Maximization of submodular functions
• Problem:
• Simplest constraint
• cardinality constraints
• for a given k, we require that
• Greedy algorithm
10
Greedy Algorithm
11
Matroid constraints
Greedy algorithm is an ½-approximation algorithm
12
Knapsack constraint
• Knapsack constraint:
• Greedy Algorithms:
• Analysis:
• Scb: the solution provided by the cost-benefit
greedy algorithm
• Suc: the solution returned by returned by the
uniform cost
[Leskovec et al. KDD 2007]
13
Speeding up the greedy algorithm
• Lazy evaluation [Leskovec et al. KDD 2007]
• First iteration as usual
• Keep an order list of marginal gain Δi from the previous
iteration
• Re-evaluate the marginal gain only for top element i
• if Δi stays on top, user it, otherwise re-sort
Sorted list in the descending
order: t, s at i-th iteration
In the (i+1)-th iteration:
14
Fast algorithms
• Summary:
• Randomized Greedy
w is the threshold
15
Badanidiyuru, Ashwinkumar, and Jan Vondrák. "Fast algorithms for
maximizing submodular functions." SODA2014.
Lazier Than Lazy Greedy
• Random Sampling
Cardinality Constraint
• Analysis: (1-1/e-ε) approximation
16
Mirzasoleiman, Baharan, Ashwinkumar Badanidiyuru, Amin Karbasi, Jan Vondrák, and
Andreas Krause. "Lazier Than Lazy Greedy." AAAI 2015.
Complex constraints
• Submodular maximization using the multilinear extension
• Submodular optimization over graphs
• the set S forms a path, or a tree on G of weight at most B
• Robust submodular optimization
• Consider adversaries (Game theory)
• Nonmonotone submodular functions
• E.g., a monotone submodular function f, and a modular cost
function c
• We want to max. non-monotone function
17
All have good approximations using Greedy based
algorithm (See Krause’s survey for details)
Online maximization of submodular functions
• The objective may not be known in advance
• Objectives functions {f1,…,fT} drawn from
some distribution
• At each round, select certain element
• Two settings:
• no-regret setting
• the choices in any round are not constrained by what one
did in previous rounds and the goal is to perform well on
average
• competitive setting
• a sequence of irrevocable decisions
• Previous round choices may affect the decision of the
current round
18
Adaptive submodularity
• We wish to adaptively select a set, observing
and taking into account feedback after
selecting any particular element.
• E.g., when placing the next sensor, adaptively
taking into account measurements provided by the
sensors selected so far
• Active learning
• …
Daniel Golovin and Andreas Krause, . Adaptive Submodularity: Theory and
Applications in Active Learning and Stochastic Optimization. Journal of
Artificial Intelligence Research (JAIR), 2011.
19
Recent Progress
Recent Progress
•
•
•
•
•
21
Submodular Welfare problem
Submodular function over integer lattice
Distributed Submodular Maximization
Streaming Submodular Maximization
Submodular Optimization with Submodular
Cover and Submodular Knapsack
Constraints
Submodular Welfare problem
• Problem:
• Algorithm:
• Continuous Greedy Algorithm provides (1-1/e-o(1))appriximation
22
Jan Vondrák. "Optimal approximation for the submodular welfare
problem in the value oracle model." STOC 2008.
Submodular Welfare problem
23
Jan Vondrák. "Optimal approximation for the submodular welfare
problem in the value oracle model." STOC 2008.
Submodular function over integer lattice
• Integer lattice: vector
• Submodular function satisfies:
• Greedy algorithm provides (1-1/e)-approx. for
the cardinality constraint (ICML 2014)
• Recent paper:
• Consider cardinality, matriod and knapsack
constraint
24
1. Soma, Tasuku, and Yuichi Yoshida. "Maximizing Submodular Functions with
the Diminishing Return Property over the Integer Lattice." arXiv preprint
arXiv:1503.01218 (2015)
2. Soma, Tasuku, Naonori Kakimura, Kazuhiro Inaba, and Ken-ichi
Kawarabayashi. "Optimal budget allocation: Theoretical guarantee and efficient
algorithm." ICML 2014.
Distributed Submodular Maximization I
• Greedy Algorithm for the submodular function
25
Mirzasoleiman, Baharan, Amin Karbasi, Rik Sarkar, and Andreas Krause. "Distributed
submodular maximization: Identifying representative elements in massive data." NIPS
2013.
Distributed Submodular Maximization II
• Summary of Greedy Algorithm in MapReduce
SPAA 2013 best paper
Kumar, Ravi, Benjamin Moseley, Sergei Vassilvitskii, and Andrea Vattani. "Fast
greedy algorithms in mapreduce and streaming." SPAA 2013
26
Streaming Submodular Maximization
• Assume elements set V is ordered
• any streaming algorithm must process V in the given order
• At each iteration t
• the algorithm maintains a memory of subset of elements Mt
points;
• and must be ready to output a candidate feasible solution St
• When a new point arrives, the algorithm may select
to remember it, and discard previous elements
Badanidiyuru, Ashwinkumar, Baharan Mirzasoleiman, Amin Karbasi,
and Andreas Krause. "Streaming submodular maximization: Massive
data summarization on the fly." KDD 2014.
27
Streaming Submodular Maximization
• Simple example
• our memory can only store one element
• Algorithm:
Badanidiyuru, Ashwinkumar, Baharan Mirzasoleiman, Amin Karbasi,
and Andreas Krause. "Streaming submodular maximization: Massive
data summarization on the fly." KDD 2014.
28
Submodular knapsack constraint
• Two types of problem
• Submodular Cost Submodular Cover (SCSC)
• Submodular Cost Submodular Knapsack
(SCSK)
• Both f and g are submodular function
29
Iyer, Rishabh K., and Jeff A. Bilmes. "Submodular optimization
with submodular cover and submodular knapsack constraints."
NIPS 2013.
References
30
1. http://submodularity.org/
2. Krause, Andreas, and Daniel Golovin. "Submodular function
maximization." Tractability: Practical Approaches to Hard Problems
3 (2012): 19.
3. Mirzasoleiman, Baharan, Ashwinkumar Badanidiyuru, Amin
Karbasi, Jan Vondrák, and Andreas Krause. "Lazier Than Lazy
Greedy." AAAI 2015.
4. Badanidiyuru, Ashwinkumar, and Jan Vondrák. "Fast algorithms for
maximizing submodular functions." SODA2014.
5. Jan Vondrák. "Optimal approximation for the submodular welfare
problem in the value oracle model." STOC 2008.
6. Soma, Tasuku, and Yuichi Yoshida. "Maximizing Submodular
Functions with the Diminishing Return Property over the Integer
Lattice." arXiv preprint arXiv:1503.01218 (2015)
7. Soma, Tasuku, Naonori Kakimura, Kazuhiro Inaba, and Ken-ichi
Kawarabayashi. "Optimal budget allocation: Theoretical guarantee
and efficient algorithm." ICML 2014.
References
8. Mirzasoleiman, Baharan, Amin Karbasi, Rik Sarkar, and Andreas
Krause. "Distributed submodular maximization: Identifying
representative elements in massive data." NIPS 2013.
9. Kumar, Ravi, Benjamin Moseley, Sergei Vassilvitskii, and Andrea
Vattani. "Fast greedy algorithms in mapreduce and streaming."
SPAA 2013.
10. Badanidiyuru, Ashwinkumar, Baharan Mirzasoleiman, Amin
Karbasi, and Andreas Krause. "Streaming submodular
maximization: Massive data summarization on the fly." KDD 2014.
11. Iyer, Rishabh K., and Jeff A. Bilmes. "Submodular optimization with
submodular cover and submodular knapsack constraints." NIPS
2013.
31