1. home and garden
2. gardening and landscaping
3. landscaping

ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
What is a strong/good (I)LP formulation?
ILP Formulation Properties and Strenghtening
Techniques
Linear program:
Version 2010.3
low number of variables
low number of constraints
Andreas M. Chwatal
complexity grows polynomially in these entities
Algorithms and Data Structures Group
Institute of Computer Graphics and Algorithms
Vienna University of Technology
VU Fortgeschrittene Algorithmen und Datenstrukturen
April 2010
1
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
2
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
What is a strong/good (I)LP formulation?
The pigeonhole
Integer Linear Program:
Suppose, we want to place n + 1 pigeons into n holes, such that each hole
contains exactly one pigeon. (This is clearly infeasible)
Let F = {x1 , . . . , xk } denote the set of feasible integer solution to
some ILP under consideration
Convex hull
conv(F) =
( k
X
i=1
Variable xij = 1 if pigeon i is in hole j, zero otherwise.
)
X
k
λi xi λi = 1, λi ≥ 0
Each hole pigeon must be placed in one of
the holes:
i=1
Let P denote the polyhedron given by the linear relaxation of the ILP:
n
X
xij = 1,
i = 1, . . . , n + 1
(1a)
j=1
conv(F) ⊆ P
xij ∈ {0, 1}
Optimal case: conv(F) = P, but this can usually not be achieved
i = 1, . . . , n + 1, j = 1, . . . , n
(1b)
In a strong formulation P closely approximates conv(F)
3
4
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
ILP Formulations
The pigeonhole (cont.)
xij ≤ 1,
k-MST
General Purpose Cutting Planes
Formulation 1:
j = 1, . . . , n, i 6= k, i, k = 1, . . . , n + 1
n+1
X
Cutting-plane separation
Given: n potential facility locations with opening costs cj , m clients with
service costs dij (for client i from facility j)
Two alternatives for the additional constraints, i.e. each hole must
contain no more than one pigeon:
xij + xkj ≤ 1,
Spanning Trees
Example: Facility Location
(2)
j = 1, . . . , n
min
(3)
s.t.
i=1
n
X
j=1
n
X
cj yj +
m X
n
X
dij xij
(4a)
i=1 j=1
xij = 1
for all i
(4b)
j=1
Linear relaxation of formulation with (2) is feasible
(with xij = 1/n, for all i, j), but infeasible with (3).
xij ≤ yj
for all i, j
(4c)
0 ≤ xij ≤ 1, yi ∈ {0, 1}
for all i, j
(4d)
5
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
ILP Formulations
Facility Location (cont.)
Formulation 2:
min
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
Spanning Trees
In this section we study various (M)ST formulations. Why?
n
X
cj yj +
m X
n
X
j=1
s.t.
6
n
X
dij xij
(5a)
Spanning tree (ST) problems arise in various contexts, often as
subproblems to more complex problems.
(5b)
While the MST is solvable in polynomial time, further constraints usually
make the problem NP-hard.
Variants:
i=1 j=1
xij = 1
for all i
xij ≤ m · yj
for all j
(5c)
for all i, j
(5d)
j=1
m
X
i=1
0 ≤ xij ≤ 1, yi ∈ {0, 1}
Which formulation is better?
F1 has n + nm constraints, whereas F2 has n + m constraints
But, PF 1 ⊂ PF 2 !! (where PX denotes the polyhedron corresponding
to the LP relaxation of formulation X )
7
Minimum (Weight) Spanning Tree
Steiner Tree, Prize Collecting ST
{Delay, Resource, Hop, Diameter, ...} - Constrained S.T.
Minimum Label Spanning Tree
...
8
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
ILP Formulations
Notation
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
Variables
In this section we consider various formulations of spanning trees as
Integer Linear Programs (ILPs)
Constants:
Graph G = (V , E ), n = |V |, m = |E |
edge-weights we > 0 for all e ∈ E
S ⊆ V,T ⊆ V
Variables:
E (S, T ) = {e = {i, j} ∈ E | i ∈ S ∧ j ∈ T }
Edge-variables xe ∈ {0, 1}, indicating if edge e is part of the solution
(xe = 1), or not (xe = 0)
Complement S¯ = V \S
¯
Cutset δ(S) = E (S, S)
Flow variables fe ≥ 0, indicating how much flow goes over edge e
Ingoing arcs: δ − (v ) = {(u, v ) | u ∈ V \{v }}
Outgoing arcs: δ + (v ) = {(v , u) | u ∈ V \{v }}
9
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
ILP Formulations
ILP-Formulation
min
X
10
Spanning Trees
xe we
(6a)
X
xe = n − 1
(6b)
Psub = {x ∈ R|E | |
General Purpose Cutting Planes
xe ≤ |S| − 1
for all S ⊂ V , S 6= ∅
X
xe = n − 1,
e∈E
X
e∈E
X
k-MST
Let Psub denote the polyhedron associated to the linear programming
relaxation of formulation (6), i.e.
e∈E
s.t.
Cutting-plane separation
Subtour Elimination Formulation
xe ≤ |S| − 1
for all S ⊂ V , S 6= ∅, 0 ≤ x ≤ 1}
e∈E (S)
(6c)
e∈E (S)
xe ∈ {0, 1}
(6d)
Subtour elimination constraints: Inequalities (6c) ensure that no subgraph
induced by S contains a cycle.
11
Theorem
The extreme points of the polyhedron Psub are the 0 − 1 incidence vectors
of spanning trees.
12
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
ILP Formulations
Spanning Trees
Cycle-Elimination Formulation
Cutset Formulation
Special case: Let Pcec denote the polyhedron resulting from the
subtour-formulation, but restricting E(S) with S ⊂ V , S 6= ∅ to cycles.
Pcut = {x ∈ R|E | |
X
Cutting-plane separation
X
xe = n−1,
e∈E
k-MST
General Purpose Cutting Planes
xe ≥ 1 for all S ⊂ V , S 6= ∅, 0 ≤ x ≤ 1}
e∈δ(S)
(7)
Theorem
Psub ⊆ Pcec .
Theorem
Psub ( Pcut .
Pcut can have fractional extreme points and is thus larger than Psub .
Question: Does Psub ( Pcec , i.e. strict inclusion hold?
13
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
14
ILP Formulations
Spanning Trees
1
k-MST
General Purpose Cutting Planes
3
1
5
0
0
2
e∈E
X
1
0
¯
E = E (S) ∪ δ(S) ∪ E (S)
P
P
¯ − 1.
If x ∈ Psub , then e∈E (S) xe ≤ |S| − 1 and e∈E (S)
¯ xe ≤ |S|
Since
X
xe = n − 1
we obtain
Cutting-plane separation
Example of fractional solution of Pcut
Proof.
For any set of nodes S we have
edge weights we
xe ≥ 1
1
1
4
1
3
1/2
e∈δ(S)
.
Hence, x ∈ Pcut and therefore Pcut ⊇ Psub .
1/2 1
By the example from the next slide we can show that the inclusion may
be strict. 2
15
1/2 0
1
1/2
edge selection variables xe
LP = 3/2 (z ILP = 2)
zcut
opt
4
1
0
5
0
1
16
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
ILP Formulations
Single Commodity Flow Formulation
min wT x
s.t.
Cutting-plane separation
k-MST
General Purpose Cutting Planes
(8a)
X
fe −
X
e∈δ + (1)
e∈δ − (1)
X
X
fe −
fe = n − 1
(8b)
for all v 6= 1, v ∈ V
(8c)
fij ≤ (n − 1)xe
fe = 1,
for every edge e = {i, j}
(8d)
fji ≤ (n − 1)xe
for every edge e = {j, i}
(8e)
for all e ∈ E
(8g)
e∈δ + (v )
e∈δ − (v )
X
Spanning Trees
Single Commodity Flow Formulation
xe = n − 1
(8f)
e∈E
f ≥ 0, xe ∈ {0, 1}
Pflow = {x ∈ R|E | | x, f satisfy Inequalities (8) without the integrality conditions}
(9)
Remark: The single commodity flow formulation just requires a
polynomial number of equations, but the corresponding polyhedron is
larger than Psub .
Theorem
Psub ( Pflow .
17
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
18
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
Properties
Example of solution of Pflow
4
1
1
3
1
1 0
1/4 1
2
1
1
0
Theorem
1
3/4
5
0
1
4
Psub ⊆ Pcut ⊆ Pflow
(10)
0
Often directed formulations yield tighter polyhedra.
1
To derive the directed formulations introduce for each edge {i, j} the arcs
(i, j) and (j, i) and associate variables ya to them and set xe = yij + yji .
Let Pdcut denote the polyhedron of the directed cut formulation:
1
Theorem
LP = 1 (z ILP = 2)
zflow
opt
Psub = Pdcut
19
(11)
20
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
Steiner Tree
Prize Collecting Steiner Tree (cont.)
Steiner Tree (ST) Problem
Let zj ∈ {0, 1} indicate if node j is included in the Steiner tree. By
extending the subtour formulation for the MST problem, we obtain:
Given a graph G = (V , E ); find a minimum weight subtree of G spanning
all terminal nodes T ⊂ V
min
X
we xe −
e∈E
The subtree might, or might not include some of the other optional
“Steiner” nodes S = V \T .
s.t.
X
X
(12a)
for all U ⊂ V and for all k ∈ U
(12b)
for all i ∈ T ∪ {r }
(12d)
xe ∈ {0, 1}
for all e ∈ E
(12e)
zi ∈ {0, 1}
for all i ∈ V
(12f)
X
zi
i∈U\{k}
X
xe =
e∈E
Given a graph G = (V , E ) with edge weights we for each e ∈ E , root
node r . For the other nodes j ∈ V \{r } we have a profit pj if the tree
contains j.
pi z i
i∈V
xe ≤
e∈E (U)
Prize Collecting Steiner Tree (PCST) Problem
X
zi
(12c)
i∈V \{r }
zi = 1
22
21
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
ILP Formulations
Prize Collecting Steiner Tree (cont.)
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
Prize Collecting Steiner Tree (cont.)
G0 = (V 0, E 0)
Determination
node:
Inequalities (12b) are called generalized subtour elimination
constraints
⇒ graph spanned by selected nodes does not contain a subtour.
of
the
root
a
Artificial root node a
Enhanced graph
G 0 = (V 0 , E 0 )
Equality (12d) ensures the correct number of edges.
V 0 = V ∪ {a}
Of course this the LP-relaxation of this formulation does not
guarantee x and z to be integer.
G = (V, E)
E 0 = E ∪ {{a, i} | i ∈ V }
Make sure, that only one of
the edges in E 0 \E is used!
23
24
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
ILP Formulations
Cycle Elimination Cuts
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
Cycle Elimination Cut Separation
Cycle-Elimination Cuts:
based on subtour-formulation, but instead of S ⊆ V we only consider
cycles C ⊆ V .
Consider each edge e = {i, j} with xe > 0. (can this be improved?)
Cycle elimination cuts can be separated by shortest-path computations
Compute a shortest path from i to j w.r.t. edge weights
we = (1 − xelp ). Here, xelp denotes the value of variable xe in the
current LP-relaxation.
X
xe ≤ |C | − 1 ⇔ |C | −
e∈E (C )
X
xe ≥ 1 ⇔
e∈E (C )
X
“delete” the edge, or set its weight to some sufficiently large constant
(1 − xe ) ≥ 1
If we found a path, and it’s value (plus wij ) is less than 1, we can
add an inequality enforcing these arcs “not to form a cycle”
e∈E (C )
25
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
26
ILP Formulations
Directed Connection Cuts, Subtour Elimination Cuts
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
Strategies
Compute minimum (s, t)-cut by max-flow algorithm
Given some root node s, this needs to be done for every node
t ∈ V \{s} as target node t
Add “most violated” cut vs. add all cuts
Add cuts “globally”, or just for the current node (and it’s children)
of the branch-and-bound tree
As each node needs to be connected
X
xe ≥ 1, S ⊂ V , S 6= ∅, s ∈ S, t ∈ V \S, s 6= t
Various improvements: nested cuts, back-cuts, ...
e∈δ + (S)
When sum over such a cut-set δ + (s) is lower than one, we have
found a cut, and can add the inequalities:
Directed connection cut:
X
Remark 1: Considering the edges/nodes in a randomized order might
improve the overall process.
xa ≥ 1
Remark 2: For many (tree) problems, the directed connection cuts show
the best performance in practice, e.g. due to their fast separation.
a∈δ + (S)
Subtour elimination:
X
xa ≤ |S| − 1 and
a∈E (S)
X
¯ −1
xa ≤ |S|
¯
a∈E (S)
27
28
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
ILP Formulations
Dual-Simplex
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
k-Node Minimum Spanning Tree (k-MST)
Given:
(undirected) graph G = (V , E , w )
nonnegative weighting function w (e) ∈ R
Purpose: effectively resolve LP after further addition of constraints
Goal: Find a minimum weight tree spanning exactly k nodes.
Starting with a dual feasible solution
Example: k = 6, z = 22
Method tries to transform into primal feasible basis while
maintaining dual feasibility
4
Operations are carried out on primal simplex tableau
12
7
8
7
13
3
2
10
5
15
6
7
8
3
4
10
4
6
6
15
6
7
29
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
30
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
Applications
Oil-field leasing: Company bought lease for an oil field. After some
period, they must return 50% of the oil-field. This part is required to be
connected!
95
25
10
60
30
40
20
15
55
70
80
45
10
20
25
Theorem (Complexity of k-MST)
The k-MST problem is NP-hard.
Proof by reduction from Steiner tree.
Programming exercise: develop a (1) flow- and a (2) cycle-elimination
or a directed cut formulation for the k-MST problem. Implement the
corresponding branch-and-bound and branch-and-cut algorithms!
For more information to the programming exercise see the seperate slides!
Company assigns profit-values to the pumps. Node-weighted
d|V |/2e-node MST (blue area) corresponds to the best connected part for
the company to return.
31
32
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
Chv´atal-Gomory (C-G) Cutting Planes
We consider the following ILP
Valid inequality: A linear inequality αT x ≤ α0 , or (α, α0 ), that is
satisfied by all points of a polyhedron is called valid inequality.
We are interested in valid inequalities w.r.t. conv(F)
max cT x
(13a)
s.t. Ax ≤ b
(13b)
We can now choose a u ∈ Rm
+ . All x satisfying (13) also satisfy
If (α, α0 ) is valid for conv(F) but not P ⇒ strenghtening the ILP
formulation
n
n
X
X
(uT Aj )xj ≤ uT b ⇒
buT Aj cxj ≤ uT b
There exist general techniques to find such inequalities
j=1
Better LP relaxations imply better bounds ⇒ speedup of
branch-and-bound methods
j=1
.. and by using the integrality of x ..
Chv´atal-Gomory Cutting Planes
n
X
buT Aj cxj ≤ buT bc
(14)
j=1
34
33
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
ILP Formulations
C-G Example [4]
max 2x1 + x2
(15a)
s.t. 7x1 + x2 ≤ 28
(15c)
− 8x1 − 9x2 ≤ −32
(15d)
x1 , x2 ∈ Z+ ∪ {0}
(15e)
1
21 , u2
=
7
22 , u3
Cutting-plane separation
k-MST
General Purpose Cutting Planes
Gomory cutting planes are particular manifestations of C-G cutting
planes
x2
Can be directly extracted from simplex tableau
P
Let x0 − kj=1 cj xj = 0. Consider the following ILP in its standard form
(including slack variables xk+1 , . . . , xn ):
1
3
⇒ −3x1 − 2x2 ≤ −9
u1 =
(15b)
− x1 + 3x2 ≤ 7
u1 = 0, u2 = 13 , u3 =
Spanning Trees
Gomory Cutting Planes
max
n
X
cj xj
(16a)
j=1
x1
s.t.
=0
n
X
aij xj = bi
i = 0, . . . , m
(16b)
j=0
⇒ x2 ≤ 3
35
36
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
Gomory Cutting Planes (cont.)
Gomory Cutting Planes (cont.)
We now introduce a partitioning of the indices, where β = (β1 , . . . , βn )
represent the basic solution indices, and η = (η1 , . . . , ηn−m ) the non-basic
indices.
A basic solution is thus given by
(18) can also be solved in terms of nonbasic variables:
∗
x =
(xβ∗1 , . . . , xβ∗m , xη∗1
=
0, . . . , xη∗n−m
= 0)
xβ∗1 , . . . , xβ∗m is a unique solution to
m
X
aiβj xβj = bi , i = 1, . . . , m
(17)
xβi +
j=1
xβi +
m
X
aiβj xβj +
j=0
n−m
X
n−m
X
j=1
A−1
β b
It follows, that
=
By this partitioning a solution to 16 is given by
¯aβi ηj xηj = xβ∗i , i = 0, . . . , m
(19)
This is exactly what we have in a row of the Simplex-tableau!!! By
rounding down the coefficients we obtain the Gomory (fractional)
cutting planes:
j=1
x∗β
n−m
X
b¯aβi ηj cxηj = bxβ∗i c, i = 0, . . . , m
(20)
By subtracting (19) from (21) we obtain an equivalent inequality
aiηj xηj = bi , i = 1, . . . , m
(18)
j=1
xβi +
n−m
X
j=1
(b¯aβi ηj c − ¯aβi ηj )xηj = bxβ∗i c − xβ∗i , i = 0, . . . , m
(21)
37
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
38
ILP Formulations
Gomory Cutting Planes
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
Gomory Cutting Planes
Example (continued). We are given the following optimal simplex table:
x0
x1
7
x3
22
3
x3
22
1
x2 +
x3
22
3
+ x3
2
5
x4
22
1
x4
22
7
+ x4
22
5
+ x4 + x5
2
21
2
7
2
7
=
2
55
=
2
+
+
=
=⇒x0
≤10
(22a)
+
−
=
=⇒x1 − x4
≤3
(22b)
=⇒x2
≤3
(22c)
=⇒x3 + 2x4 + x5
≤27
(22d)
x2
Drawbacks:
Large LPs ⇒ long running times in each iteration
Many non-zero coefficients ⇒ numeric issues
x1
39
40
ILP Formulations
Spanning Trees
Cutting-plane separation
k-MST
General Purpose Cutting Planes
Literature
1
2
3
4
Bertsimas, Weismantel, “Optimization over Integers”,
Dynamic Ideas, 2005
Magnanti, Wolsey, “Optimal Trees”, 1994
Nemhauser, Wolsey, “Integer and Combinatorial
Optimization”, Wiley, 1999
Lee, “A First Course in Combinatorial Optimization”,
Cambridge University Press, 2004
41