E0 249: Approximation Algorithms
Spring 2015
Lecture 11: March 20
Lecturer: Arnab Bhattacharyya
11.1
Scribe: Amleshwar Kumar and Nitin Singh
Preliminaries
An n×n real matrix M is called symmetric if Mij = Mji for all i, j ∈ [n]. Let SYMn denote
P the set of all n×n
real symmetric matrices. For matrices A, X ∈ SYMn , we define the product A·X to be i,j∈[n] aij xij , where
aij and xij denote the (i, j)th entries of A and X respectively. A real symmetric matrix X is called positive
semidefinite if all its eigen values are non-negative (note that all eigen values are real in any case). From the
standard linear algebra (which we will not go into here), we have the following equivalent characterizations
of positive semidefinite matrices:
Proposition 11.1. Let X be a real symmetric n × n matrix. Then the following are equivalent:
(i) All eigen values of X are non-negative.
(ii) For all a ∈ Rn , aT Xa ≥ 0.
(iii) X = U T U for some n × n matrix U .
A semidefinite program is an optimization problem of the form:
Maximize:
s.t.
C.X
A1 · X
A2 · X
..
.
=
=
b1
b2
An · X
X
=
bn
0.
where C, A1 , . . . , An are real symmetric matrices and b1 , . . . , bn are real numbers. The constraint X 0 in
the above denotes that X is positive semidefinite matrix.
11.2
MAX-CUT Problem
In this section we illustrate an application of semidefinite programming to obtain an approximation algorithm
for the MAX-CUT problem. Following is the MAX-CUT problem.
MAX-CUT: Given a graph G = (V, E), find S ⊆ V such that the cardinality of the set {e : e = (u, v) with u ∈
S, v 6∈ S} is maximized.
As a first approach we consider the following program:
M C1:
max
X 1 − xi xj
2
(i,j)∈E
s.t
xi ∈ {±1} ∀i ∈ [n].
11-1
11-2
Lecture 11: March 20
Algorithm 1 MAX-CUT
Require: Graph G = (V, E).
Solve SDP -A for vectors u1 , . . . , un .
Infer set S from vectors u1 , . . . , un using GW -rounding.
return S.
The solution to the above corresponds to the set S = {i : xi = 1}. It can be seen through this correspondence
that OP T ≤ M C1. Below we consider a semidefinite program, where instead of reals xi we use vectors
ui ∈ Rn .
SDP -A:
X 1 − ui · uj
2
max
(i,j)∈E
s.t
ui · ui = 1 ∀i ∈ [n].
Before we describe how a solution to SDP -A corresponds to a solution to MAX-CUT, let us observe that
OP T ≤ SDP -A. Why ? (Hint: M C1 ≤ SDP -A via embedding scalars into vectors in Rn ). We call the
vector program SDP -A to be the vector relaxation of the program M C1.
From Proposition 11.1, it follows that SDP -A is equivalent to the following semidefinite program:
SDP -B:
max
X 1 − yij
2
(i,j)∈E
s.t
yii = 1 ∀i ∈ [n]
Y 0.
Once we have a solution to SDP -B, we can obtain a solution to SDP -A by determining U such that
Y = U T U (Cholesky’s decomposition 1 ). We still have not specified how to obtain a cut from a solution to
SDP -A. This is accomplished by a procedure called GW -rounding, introduced by Goemans and Williamson.
GW -rounding:
(i) Pick a random vector s ∈ Rn with |s| = 1.
(ii) Put i in S if ui .s ≥ 0.
(iii) Let xi = 1 if ui .s ≥ 0, and xi = −1 if ui .s < 0.
Algorithm 1 contains the complete algorithm for MAX-CUT.
Analysis: Clearly, the size of the cut returned by the above algorithm is equal to the number of pairs
(i, j) : i < j such that xi 6= xj . Let pij denote the probability that xi 6= xj for i < j. Note that
the randomness comes from randomly choosing the reference vector s (It is worth pondering, how do we
randomly choose a unit vector in Rn ). Let C(s) denote the size of cut obtained for a choice of s. Then we
have,
X
E[C(s)] =
pij
(11.1)
(i,j)∈E
1 Note that since U may have irrational entries, we may not get Y = U T U exactly. But we can get arbitrarily accurate, and
the error is absorbed in the approximation factor.
Lecture 11: March 20
11-3
Now let α be the angle between vectors ui and uj . When do ui .s and uj .s have different signs ? Since
ui · s = ui · sˆ and uj · s = uj · sˆ where sˆ is the projection of s onto the plane determined by ui and uj , we
essentially need the probability that for a random vector v in the plane of ui and uj , the dot products ui · v
and uj · v have different signs. We leave it to the reader to show that it is α/π where cos α = ui .uj . Then
from (11.1) we have,
E[C(s)] =
X cos−1 (ui · uj )
π
(i,j)∈E
≥ 0.878567
X 1 − ui · uj
2
(i,j)∈E
≥ 0.878567 · SDP -A
≥ 0.878567 · OP T
In the above, we have used the fact that
11.3
2 cos−1 z
≥ 0.878567 for z ∈ [−1, 1].
π(1 − z)
How do we solve SDPs
In this section we look at the question of solving SDPs. One way is to use the following property of the
ellipsoid algorithm.
Given a convex set C contained in a ball of radius R, and a polynomial time seperation oracle (to decide if a
solution is feasible, or produce a certificate of infeasibility), ellipsoid algorithm minimizes any linear function
over C.
(∗)
We need to verify two things:
(i) Is the set of all feasible solutions to an SDP convex ?
(ii) Is there a polynomial time seperation oracle to check feasibility of a solution ?
Convexity of the set of feasible solution follows from Proposition 11.1. Let us look at the question of having
a polynomial time seperation oracle. If a solution X is infeasible, one of the following must happen.
(i) X is not symmetric. In this case there exist i, j such that Xij > Xji . Then the seperating hyperplane
is Yij ≤ Yji .
(ii) X is not positive semidefinite. In this case, there is a negative eigen value, say λ. Let a be the
corresponding eigen vector. Then we see that aT Xa < 0. The seperating hyperplane is then aT ya ≥ 0.
(iii) X violates one of the constraints. In this case, the constraint itself yields the seperating hyperplane.
One also needs to check that the feasible solutions of the SDP have bounded Frobenius norm (`2 norm of
the matrix viewed as an n2 -length vector). This is not true for arbitrary polynomial size SDP’s, but for
most applications, such as MAX-CUT above, it is true, because each entry of a feasible solution matrix
corresponds to the inner product between bounded length vectors.
11-4
11.4
Lecture 11: March 20
MAX-2SAT Problem
The following is the MAX-2SAT problem: Given a boolean clause C = X1 ∧ X2 ∧ · · · ∧ Xk where each Xi
is an ’or’ of atmost two variables. The goal is to find a {T, F } assignment of variables which maximizes the
number of clauses Xi evaluating to T .
We consider the following SDP formulation to the above. Let v1 , . . . , vn be the variables appearing in the
clauses. Let y0 ∈ {±1} be a reference variable, and y1 , . . . , yn ∈ {±1} be “SDP” variables corresponding to
boolean variables v1 , . . . , vn . We infer vi to be true when yi = y0 and false otherwise. Define:
τ (vi ) :=
1 − y0 yi
1 + y0 yi
and τ (¯
vi ) :=
.
2
2
(11.2)
Notice that for single variable clause V , τ (V ) = 1 when V is true and τ (V ) = 0 when V is false. Similarly
for a two variable clause vi ∨ vj , define:
τ (vi ∨ vj ) := 1 − τ (¯
vi )τ (¯
vj )
1 − y0 yi 1 − y0 yj
·
:= 1 −
2
2
1 + y0 yi
1 + y0 yj
1 − yi yj
:=
+
−
.
4
4
4
In the above, we use the fact that y02 = 1. Again, τ (vi ∨ vj ) is 1 or 0 according to whether vi ∨ vj is true
or false. We can come up with similar expressions for other two variable clauses, in terms of (1 − yi yj ) or
Pk
(1 + yi yj ). Thus the number of satisfied clauses is given by i=1 τ (Xi ). We thus have the following SDP:
max
X
aij (1 + yi yj ) + bij (1 − yi yj )
0≤i<j≤n
s.t
yi ∈ {±1} ∀ 0 ≤ i ≤ n.
Again, relaxing the above program to a vector program, and using the same algorithm as MAX-CUT, we
get a 0.87856-approximate algorithm for MAX-2SAT (exercise).