ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
What is a strong/good (I)LP formulation?
Linear program:
low number of variables
low number of constraints
complexity grows polynomially in these entities
ILP Formulation Properties and Strenghtening
Techniques
Version 2009.2
Integer Linear Program:
Let F = {x1 , . . . , xk } be an feasible integer solution to some ILP
Convex hull
( k
)
X
X
k
conv(F) =
λi xi λi = 1, λi ≥ 0
Andreas M. Chwatal
Algorithms and Data Structures Group
Institute of Computer Graphics and Algorithms
Vienna University of Technology
i=1
i=1
Let P denote the polyhedron associated to the linear relaxation of
the ILP:
conv(F) ⊆ P
VU Fortgeschrittene Algorithmen und Datenstrukturen
Mai 2009
Optimal case: conv(F) = P, but this is often not achievable
In a strong formulation P closely approximates conv(F)
2
1
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
ILP Formulations
The pigeonhole
Suppose, we want to place n + 1 items into n holes, such that each hole
contains exactly one item. (This is clearly infeasible)
n
X
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
Example: Facility Location
xij = 1,
Given: n potential facility locations with opening costs cj , m clients with
service costs dij (for client i from facility j)
i = 1, . . . , n + 1
(1a)
i = 1, . . . , n + 1, j = 1, . . . , n
(1b)
j=1
xij ∈ {0, 1}
Formulation 1:
min
Two alternatives for the additional constraints:
xij + xkj ≤ 1,
j = 1, . . . , n, i 6= k, i, k = 1, . . . , n + 1
(2)
s.t.
n
X
j=1
n
X
cj yj +
m X
n
X
dij xij
(4a)
i=1 j=1
xij = 1
for all i
(4b)
j=1
n+1
X
xij ≤ 1,
j = 1, . . . , n
(3)
i=1
xij ≤ yj
for all i, j
(4c)
0 ≤ xij ≤ 1, yi ∈ {0, 1}
for all i, j
(4d)
Linear relaxation of formulation with (2) is feasible (with xij = 1/n), but
infeasible with (3).
3
4
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
ILP Formulations
Facility Location (cont.)
Formulation 2:
min
Cutting-plane separation
k-MST
C/G Cutting Planes
In this section we study various ST formulations. Why?
n
X
cj yj +
m X
n
X
j=1
s.t.
Spanning Trees
Spanning Trees
n
X
dij xij
(5a)
Spanning tree (ST) problems arise in various contexts, often as
subproblems to more complex problems.
(5b)
While the MST is solvable in polynomial time, further constraints usually
make the problem NP-hard.
for all j
(5c)
Variants:
for all i, j
(5d)
i=1 j=1
xij = 1
for all i
j=1
m
X
xij ≤ m · yj
i=1
0 ≤ xij ≤ 1, yi ∈ {0, 1}
Which formulation is better?
F1 has n + nm constraints, whereas F2 has n + m constraints
But, PF 1 ⊂ PF 2 !! (where PX denotes the polyhedron corresponding
to the LP relaxation of formulation X )
Minimum (Weight) Spanning Tree
Steiner Tree, Price Collecting ST
{Delay, Resource, Hop, Diameter, ...} - Constrained S.T.
Minimum Label Spanning Tree
...
5
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
6
ILP Formulations
Notation
Variables
In this section we consider various formulations of spanning trees as
Integer Linear Programs (ILPs)
Constants:
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
edge-weights we > 0 ∀e ∈ E
Graph G = (V , E ), n = |V |, m = |E |
S ⊆ V,T ⊆ V
Variables:
E (S, T ) = {e = {i, j} ∈ E | i ∈ S ∧ j ∈ T }
edge-variables xe ∈ {0, 1}, indicating if edge e is part of the solution
(xe = 1), or not (xe = 0)
Complement S¯ = V \S
¯
Cutset δ(S) = E (S, S)
flow variables fe ≥ 0, indicating how much flow goes over edge e
7
8
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
ILP Formulations
ILP-Formulation
min
Spanning Trees
X
xe we
(6a)
k-MST
C/G Cutting Planes
xe = n − 1
(6b)
Let Psub denote the polyhedron associated to the linear programming
relaxation of formulation (6), i.e.
e∈E
s.t.
Cutting-plane separation
Subtour elimination formulation
X
Psub = {~x ∈ R|E | |
X
xe ≤ |S| − 1 ∀S ⊂ V , S 6= ∅
X
xe = n − 1,
e∈E
X
e∈E
xe = |S| − 1
for all ∅ =
6 S ⊂ V , ~0 ≤ ~x ≤ ~1}
e∈E (S)
(6c)
e∈E (S)
xe ∈ {0, 1}
(6d)
Theorem
The extreme points of the polyhedron Psub are the 0 − 1 incidence vectors
of spanning trees.
Subtour elimination constraints: inequalities 6c ensure that no subgraph
induced by S contains a cycle.
9
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
10
ILP Formulations
Cycle-elimination formulation
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
Cutset formulation
Special case: Let Pcec denote the polyhedron resulting from the
subtour-formulation, but restricting E(S) with ∅ =
6 S ⊂ V to cycles.
Pcut = {~x ∈ R|E | |
X
e∈E
xe = n − 1,
X
xe ≥ 1 for all ∅ =
6 S ⊂ V , ~0 ≤ ~x ≤ ~1}
e∈δ(S)
(7)
Theorem
Psub ⊆ Pcec .
Theorem
Pcut ⊇ Psub .
Pcut can have fractional extreme points and is thus larger than Psub .
Question: Does Psub ⊂ Pcec hold?
11
12
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
ILP Formulations
1
X
1
0
2
edge weights we
xe ≥ 1
1
1
2
13
k-MST
C/G Cutting Planes
~ T ~x
min w
1/2 0
1
4
1
0
5
0
1
edge selection variables xe
LP = 3/2 (z ILP = 2)
zcut
opt
Spanning Trees
Cutting-plane separation
14
k-MST
C/G Cutting Planes
Single Commodity Flow Formulation
(8a)
fe −
e∈δ + (1)
X
e∈δ − (v )
X
fe = n − 1
(8b)
fe = 1, for all v 6= 1, v ∈ V
(8c)
e∈δ − (1)
fe −
X
e∈δ + (v )
fij ≤ (n − 1)xe for every edge e = {i, j}
(8d)
fji ≤ (n − 1)xe for every edge e = {j, i}
(8e)
X
3
1/2
ILP Formulations
Single Commodity Flow Formulation
X
1
1/2 1
By the example from the next slide we can show that the inclusion may
be strict. Cutting-plane separation
4
1/2
.
Hence, ~x ∈ Pcut and therefore Pcut ⊇ Psub .
s.t.
C/G Cutting Planes
5
0
e∈δ(S)
Spanning Trees
k-MST
3
1
e∈E
ILP Formulations
Cutting-plane separation
0
¯
E = E (S) ∪ δ(S) ∪ E (S)
P
P
¯ − 1.
If ~x ∈ Psub , then e∈E (S) xe ≤ |S| − 1 and e∈E (S)
¯ xe ≤ |S|
Since
X
=n−1
we obtain
Spanning Trees
Example of fractional solution of Pcut
Proof.
For any set of nodes S we have
xe = n − 1
(8f)
e∈E
~f ≥ ~0, 0 ≤ xe ≤ 1 and integral for every edge e ∈ E
Pflow = {~x ∈ R|E | | ~x , ~f satisfy Inequalities 8 without the integrality conditions}
(9)
Remark: The single commodity flow formulation just requires a
polynomial number of equations, but the corresponding polyhedron is
larger than Psub .
Theorem
Pflow ⊃ Psub .
(8g)
15
16
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
Properties
Example of solution of Pflow
4
1
1
3
1
1 0
1/4 1
2
1
1
0
Theorem
1
3/4
Psub ⊆ Pcut ⊆ Pflow
5
0
Often directed formulations yield tighter polyhedra.
1
1
4
(10)
0
To derive the directed formulations introduce for each edge {i, j} the arcs
(i, j) and (j, i) and associate variables ya to them and set xe = yij + yji .
Let Pdcut denote the polyhedron of the directed cut formulation:
1
Theorem
LP = 1 (z ILP = 2)
zflow
opt
Psub = Pdcut
(11)
18
17
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
Steiner Tree
Price Collecting Steiner Tree (cont.)
Steiner Tree (ST) Problem
Let zj ∈ {0, 1} indicate if node j is in the Steiner tree. We can extend the
subtour formulation:
X
X
min
we xe −
pi zi
(12a)
Given a graph G = (V , E ); find a minimum weight subtree of G spanning
all terminal nodes T ⊂ V
e∈E
s.t.
The subtree might, or might not include some of the other optional
“Steiner” nodes S = V \T .
X
i∈V
xe ≤
e∈E (U)
X
xe =
e∈E
Price Collecting Steiner Tree (PCST) Problem
Given a graph G = (V , E ) with edge weights we for each e ∈ E , root
node r . For the other nodes j ∈ V \{r } we have a profit pj if the tree
contains j.
19
X
zi
for all U ⊂ V and f.a. k ∈ U
(12b)
i∈U\{k}
X
zi
(12c)
i∈V \{r }
zr = 1
(12d)
0 ≤ xe ≤ 1, 0 ≤ zi ≤ 1
(12e)
Remark: Inequalities (12b) are called generalized subtour elimination
constraints
Remark: Of course this formulation does not guarantee ~x and ~z to be
integer.
20
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
ILP Formulations
Price Collecting Steiner Tree (cont.)
of
the
Cutting-plane separation
k-MST
C/G Cutting Planes
Cycle-Elimination Cuts:
based on subtour-formulation, but instead of S ⊆ V we only consider
cycles C ⊆ V .
G0 = (V 0, E 0)
Determination
node:
Spanning Trees
Cycle Elimination Cuts
root
Cycle elimination cuts can be separated by shortest-path computations
a
X
artificial root node a
enhanced graph
G 0 = (V 0 , E 0 )
xe ≤ |C | − 1 ⇔ |C | −
e∈δ(C )
X
X
xe ≥ 1 ⇔
e∈δ(C )
(1 − xe ) ≥ 1
e∈δ(C )
G = (V, E)
V 0 = V ∪ {a}
Cut Separation by shortest path computations
E 0 = E ∪ {{a, i} | i ∈ V }
consider each edge e = {i, j} with xe > 0. (can this be improved?)
make sure, that only one of
the edges in E 0 \E is used!
“delete” the edge, or set its weight to some sufficiently large constant
compute a shortest path from i to j
if we found a path, and it’s value (plus wij ) is less than 1, we can
add an inequality enforcing these arcs “not to form a cycle”
21
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
22
ILP Formulations
Directed Connection Cuts, Cycle Elimination Cuts
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
Strategies
Add “most violated” cut vs. add all cuts
Compute minimum (s, t)-cut by max-flow algorithm
Add cuts “globally”, or just for the current node (and it’s children)
of the branch-and-bound tree
Given some root node s, this needs to be done for every node
t ∈ V \{s} as target node t
Various improvements: nested cuts, back-cuts, ...
As
P each node needs to be connected
6 S ⊂ V , s ∈ S, t ∈ V \S, s 6= t
e∈δ + (S) xe ≥ 1, ∅ =
Remark 1: Considering the edges/nodes in a randomized order might
improve the overall process.
When sum over such a cut-set δ + (s) is lower than one, we have
found a cut
Remark 2: For many (tree) problems, the directed connection cuts show
the best performance in practice, e.g. due to their fast separation.
23
24
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
ILP Formulations
Dual-Simplex
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
k-node minimum spanning tree (k-MST)
Given:
(undirected) graph G = (V , E , w )
nonnegative weighting function w (e) ∈ R
Purpose: effectively resolve LP after further addition of constraints
Goal: Find a minimum weight tree spanning exactly k nodes.
Starting with an dual feasible solution
Example: k = 6, z = 22
Method tries to transform into primal feasible basis while
maintaining dual feasibility
4
Operations are carried out on primal simplex tableau
12
7
8
7
13
3
2
10
5
15
6
7
8
3
4
10
4
6
15
6
6
7
25
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
26
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
Applications
Oil-field leasing: Company bought lease for an oil field. After some
period, they must return 50% of the oil-field. This part is required to be
connected!
95
25
10
60
30
40
20
15
55
70
80
45
10
20
25
Theorem (Complexity of k-MST)
The k-MST problem is NP-hard.
Proof by reduction from Steiner tree.
Programming exercise: develop a (1) flow- and a (2) cycle-elimination
or a directed cut formulation for the k-MST problem. Implement the
corresponding branch-and-bound and branch-and-cut algorithms!
For more information to the programming exercise see the seperate slides!
Company assigns profit-values to the pumps. Node-weighted
d|V |/2e-node MST (blue area) corresponds to the best connected part for
the company to return.
27
28
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
Chv´atal-Gomory Cutting Planes
We consider the following ILP
Valid inequalities: (linear) inequalities, that are satisfied by all
integer points of a formulation
⇒ strenghtening the formulation
max cT x
(13a)
s.t. Ax ≤ b
(13b)
We can now choose a u ∈ Rm
+ . All x satisfying (13) also satisfy
There exist general techniques to find such inequalities
n
n
X
X
(uT Aj )xj ≤ uT b ⇒
buT Aj cxj ≤ uT b
Better LP relaxations imply better bounds ⇒ speedup of
branch-and-bound methods
j=1
j=1
.. and by using the integrality of x ..
Chv´atal-Gomory Cutting Planes
n
X
buT Aj cxj ≤ buT bc
(14)
j=1
30
29
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
ILP Formulations
C-G Example [4]
max 2x1 + x2
(15a)
s.t. 7x1 + x2 ≤ 28
(15c)
− 8x1 − 9x2 ≤ −32
(15d)
x1 , x2 ∈ Z+ ∪ {0}
(15e)
1
21 , u2
=
7
22 , u3
Cutting-plane separation
k-MST
C/G Cutting Planes
Gomory cutting planes are particular manifestations of C-G cutting
planes
x2
Can be directly extracted from simplex tableau
P
Let x0 − kj=1 cj xj = 0. Consider the following ILP in its standard form
(including slack variables xk+1 , . . . , xn ):
1
3
⇒ −3x1 − 2x2 ≤ −9
u1 =
(15b)
− x1 + 3x2 ≤ 7
u1 = 0, u2 = 13 , u3 =
Spanning Trees
Gomory Cutting Planes
max
n
X
cj xj
(16a)
j=1
x1
s.t.
=0
n
X
aij xj = bi
i = 0, . . . , m
(16b)
j=0
⇒ x2 ≤ 3
31
32
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
Gomory Cutting Planes (cont.)
Gomory Cutting Planes (cont.)
~ = (β1 , . . . , βn )
We now introduce a partitioning of the indices, where beta
represent the basic solution indices, and ~η = (η1 , . . . , ηn−m ) the non-basic
indices.
A basic solution is thus given by
(18) can also be solved in terms of nonbasic variables:
∗
~x =
(xβ∗1 , . . . , xβ∗m , xη∗1
=
0, . . . , xη∗n−m
= 0)
xβ∗1 , . . . , xβ∗m is a unique solution to
m
X
aiβj xβj = bi , i = 1, . . . , m
(17)
xβi +
j=1
xβi +
~
A−1
β b
m
X
aiβj xβj +
j=0
n−m
X
n−m
X
j=1
It follows, that
=
By this partitioning a solution to 16 is given by
¯aβi ηj xηj = xβ∗i , i = 0, . . . , m
(19)
This is exactly what we have in a row of the Simplex-tableau!!! By
rounding down the coefficients we obtain the Gomory (fractional)
cutting planes:
j=1
~xβ∗
n−m
X
C/G Cutting Planes
b¯aβi ηj cxηj = bxβ∗i c, i = 0, . . . , m
(20)
By subtracting (19) from (21) we obtain an equivalent inequality
aiηj xηj = bi , i = 1, . . . , m
(18)
j=1
xβi +
n−m
X
j=1
(b¯aβi ηj c − ¯aβi ηj )xηj = bxβ∗i c − xβ∗i , i = 0, . . . , m
(21)
33
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
34
ILP Formulations
Gomory Cutting Planes
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
Gomory Cutting Planes
Example (continued). We are given the following optimal simplex table:
x0
x1
7
x3
22
3
x3
22
1
x2 +
x3
22
3
+ x3
2
5
x4
22
1
x4
22
7
+ x4
22
5
+ x4 + x5
2
21
2
7
2
7
=
2
55
=
2
+
+
=
=⇒x0
≤10
(22a)
+
−
=
=⇒x1 − x4
≤3
(22b)
=⇒x2
≤3
(22c)
=⇒x3 + 2x4 + x5
≤27
(22d)
x2
Drawbacks:
Large LPs ⇒ long running times in each iteration
Many non-zero coefficients ⇒ numeric issues
x1
35
36
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
C/G Cutting Planes
Literature
1
2
3
4
Bertsimas, Weismantel, “Optimization over Integers”,
Dynamic Ideas, 2005
Magnanti, Wolsey, “Optimal Trees”, 1994
Nemhauser, Wolsey, “Integer and Combinatorial
Optimization”, Wiley, 1999
Lee, “A First Course in Combinatorial Optimization”,
Cambridge University Press, 2004
37