Combinatorial Optimization at Work: How to Find the Needle in the Haystack? Martin Oellrich, T-Systems Nova GmbH, Berlin, Germany [email protected] Abstract Telecommunications network design most often aims at specified technical solutions at the lowest possible cost. Discrete Mathematics can greatly help achieving this goal. The technical system is modelled in mathematical language and solved by suitable methods. It lies in the nature of those problems that they have a vast multitude of possible solutions. Singling out one with lowest cost among all of them is just like searching for a needle in a haystack. As it is a typical example for a telecommunication design problem, we will work out step-by-step the solution process of the Disjoint Routing Problem of Virtual Private Networks (VPN): a number of given demands must be routed through a given network such that their paths do not share any common nodes or links, and cause minimal cost among all possible solutions. 1 Introduction A Virtual Private Network (VPN) is a network that appears to be exclusively accessible and operated by a customer company alone. In physical reality, it consists of a number of permanent lines that are leased from a carrier. So control of its performance — in particular: its outages — also lies with the latter. And for this reason, the carrier has provide these lines subject to a reliability demand: one single failure in the underlying network may cause at most one single failure on the VPN level, i.e., in the customer’s netFigure 2: Virtual Private Network work. Technically speaking, the different leased lines must be routed disjointly from each other. In order to be competitive, the carrier must also consider As the computed output data, we expect a list containthe cost side of this requirement. So naturally, a disjoint routing at minimum cost is wanted. The problem of finding ing one simple routing path for each end-to-end pair of the VPN, such that this optimum is the topic of this tutorial. To make things more precise, here are the needed input • no two paths share a common trunk data: (trunk-disjoint routing), see figure 3, and • the carrier’s network, see figure 1, Figure 1: carrier’s network • with a cost coefficient on every trunk, • the Virtual Private Network, see figure 2. Figure 3: disjoint routing • the sum of the costs of all paths are at the possible minimum. Mathematically speaking, the problem is called Minimum Cost Disjoint Embedding Problem. In stating it, we have in mind to solve problem instances with a few thousand nodes and trunks. If node-disjoint routing is desired, one only has to split all nodes into two nodes that are connected by an artificial trunk with zero cost, see figure 4. Figure 4: node splitting technique Our way through the solution process will start out with a very simple but instructive example for a combinatorial search. 2 and per column. Otherwise there would be a pair that could attack each other. The second goal is the optimization goal. It tells us how to quantify (and thus to compare) the feasible objects and which extremal property we want to achieve. The quantification is called the objective function. It may be minimized or maximized. In our case, the objective function is the sum of the values the rooks are standing on. See figure 6 for a feasible solution and its objective function value. 9 7 8 2 2 3 7 4 4 1 5 6 4 1 6 3 Combinatorial Optimization We leave the context of telecommunications planning for the duration of this introduction. After we have understood the ideas and techniques, solving our original problem will be a straight-forward application. It is described in section 5. 9 2 4 4 7 3 1 1 8 7 5 6 2 4 6 3 Figure 5: a grid with numbers Figure 6: a feasible solution with value 21 How many feasible solutions are there? Given the empty grid, we have four distinct possibilities to place a rook in the first row. Subsequently, this leaves three possibilities to place one in the second row, and so on. In total, there are 4 · 3 · 2 · 1 = 24 feasible solutions. This particular problem could still be solved by hand, but let us think big. If we imagine that the setting is a real 8-by-8 chess board, there are already 40320 feasible solutions. How about a side length of 20, 50, 100? The numbers of feasible solutions explode: at a side length of about 35 we have roughly as many feasible solutions as we have particles in our universe. Almost all Combinatorial Optimization problems share a common property: they possess a vast multitude of feasible solutions and the task is to seek out one of them with the desired extremal property. This mostly involves comparing the objective values of all feasible solutions. This demonstrates the necessity of special solution techniques. Let us now consider the following optimization problem. We are given a 4-by-4 grid — one may imagine one quarter of a chess board — where every cell carries a given 2.1 Enumeration tree value (see figure 5). The task to solve: take four chess rooks As a first step, we organize the search for the optimal sound place them on the grid so as to simultaneously achieve lution in a systematic way. We enumerate all the solutwo goals: tions without repetition as follows. We imitate the way of counting the solution (see above) and start with an empty 1. no pair of the rooks may attack each other, grid. On every cell in the top row, we can place our first 2. the sum of the numbers the rooks are standing on is a rook, yielding four mutually exclusive so-called partial solutions. With each of them, we can now iterate the process minimum among all possible solutions. and place another rook on the second row wherever it is not The first goal is called a feasibility constraint, since it put below the first rook. This generates another 4 · 3 = 12 describes which combinatorial objects are considered per- mutually exclusive partial solutions with two rooks each. missible or feasible. In our case here, it declares only such This goes on until all four rooks have been placed in all solutions feasible where we have exactly one rook per row possible ways. 16 14 14 10 11 14 15 21 22 25 22 18 16 16 17 13 19 10 18 20 23 22 14 13 24 20 20 18 17 17 16 12 111111 000 000111 000 000111 111 000111 000 000111 111 000111 000 000111 111 000111 000 000111 111 000111 000 000111 111 000111 000 111111 000 000 000111 111 000 000111 111 000 000111 111 000 000111 111 000 000111 111 000 9 Figure 8: (partial) objective values in the tree 1111 0000 0000 1111 0000 1111 0000 1111 0000 1111 0000 1111 111111 000 000 000111 111 000 000111 111 000 000111 111 000 000111 111 000 000111 111 000 1111 0000 0000 1111 0000 1111 0000 1111 0000 1111 0000 1111 0001111 111 0000 1111111 000 0000 0001111 111 0000 0001111 111 0000 0001111 111 0000 0001111 111 0000 0001111 111 0000 0001111 111 0000 00000 11111 11111 00000 00000 11111 00000 11111 00000 11111 00000 11111 00000 11111 00000 11111 16 15 9 111 000 000 111 000 111 000 111 000 111 000 111 1111111 0000000 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 0000000 1111111 20 15 16 14 10 7 11 00 00 11 00 11 00 11 00 11 00 11 11 00 00 11 00 11 00 11 00 11 00 11 11 00 00 11 00 11 00 11 00 11 00 11 000 111 111 000 000 111 000 111 000 111 000 111 000 111 000 111 00 11 11 00 00 11 00 11 00 11 00 11 00 11 00 11 111 000 000 111 000 111 000 111 000 111 000 111 11 12 0 15 111 000 000 111 000 111 000 111 000 111 000 111 10 22 19 15 8 11 00 00 11 00 11 00 11 00 11 00 11 11 00 00 11 00 11 00 11 00 11 00 11 0000 1111 1111 0000 0000 1111 0000 1111 0000 1111 0000 1111 0000 1111 0000 1111 21 16 111 000 000 111 000 111 000 111 000 111 000 111 12 17 9 5 10 5 4 2 9 13 9 10 This generation process is illustrated in figure 7. The empty grid is the root of a tree which branches out with the possibilities of extending the partial solutions as said above. Figure 7: full enumeration tree 2.2 Pruning the tree As we see from figure 8, only very few vertices in the tree are actually interesting. The optimum is located within a small portion of the tree and several branches of it were a successless effort in the enumeration. If we are able to direct the search better, we can save considerable time. Keeping this in mind, we can rewrite the figure, showing Here is a first idea how to identify a vertex — and with only the objective function values of the (partial) solutions, it its entire subtree below — that does not justify further see figure 8. exploration. Remember: during the enumeration, the latSo basically, what we do is to generate the entire tree, est bold framed vertex indicates the best currently known and to single out one of its bottom vertices with minimum solution. If we now produce a vertex with a value not less value. Assuming that the bottom row of the tree is gener- than this solution value, we can stop extending the tree at ated from left to right — a way of traversion called depth this point. As the values cannot decrease when more rooks first search — figure 8 also shows the improvements made are placed, we will never find a better solution below this in the search. Every time a vertex with lower value than all vertex. Figure 9 shows the tree when this rule is observed. the ones to its left is encountered, it is marked by a bold frame. The rightmost bold framed vertex, the one with the value 10, is the desired optimum. So the way of cutting off subtrees is governed by numbers that are available along the process. This idea is strengthened in the next section. 10 11 10 14 15 16 12 17 5 9 5 4 15 9 10 13 9 2 8 For an example, consider figure 10. Since we know that every row of the grid must host exactly one rook, this one rook must stand on a cell carrying a value at least as high as the minimum value in that row. So every feasible solution must have an objective value at least the sum of these row minima. In our 4-by-4 example, we compute a lower bound of 6. How can it help us? A simple calculation like the one in figure 10 can be easily carried out in every new tree vertex. So when deciding whether to explore or not, we have three numbers at our disposal: 16 • the value curbest of the current best solution, • the value lbound of the computed lower bound. 11 12 16 0 15 10 • the value parval of the partial solution reached, and 19 20 13 18 13 14 10 9 15 14 7 10 16 15 The lower bound guarantees that no feasible solution has a lower value. So in the new vertex, no feasible solution extending the partial solution there has a lower value less than parval + lbound. Comparing this sum to the best known solution, we get an improved pruning criterion: parval + lbound ≥ curbest ⇒ prune vertex. 20 18 24 20 12 17 17 9 16 22 2.4 Improved lower bounds Figure 9: a first pruning idea The pruning criterion applies to any lower bound, not only the simple one presented in figure 10. How can we computer better ones? Surprisingly, there is a way of strengening the lower bound, often even up to the optimum value. Consider the following algorithm, illustrated in figure 11. 2.3 Lower Bounds The tree can be pruned in precisely that moment when we are sure that the exploration of a subtree cannot yield a better solution than the best one currently known. This observation also holds true if we support the pruning decision by a so-called lower bound. This is the name for a value which offers a guarantee that no feasible solution can have a smaller objective value. 9 2 4 4 7 3 1 1 8 7 5 6 2 4 6 3 2 2 1 1 6 row minima bound Figure 10: a simple lower bound 3. add per col. 9 2 4 4 0 9 2 4 4 7 3 1 1 2 9 5 3 3 8 7 5 6 −2 6 5 3 4 (n j = 1 2 0 2 original grid 4 6 1. arbitrary 3 numbers 1 −1 2.tivenegasum 3 3 4. row 5 2 minima 7 3 4 3 1) 10 5. bound Figure 11: a stronger lower bound 2.5 Correctness of the algorithm Lower Bound Algorithm 1. Write an arbitrary number over each column of the grid. 2. Add these numbers and write their negative sum on their right. This paragraph is more mathematical, as it has to prove the correctness of the algorithm stated above. The more practical oriented reader may skip it. In order to see why the algorithm always delivers a lower bound, we need to formalize our rooks problem. We give the value located in the grid cell (i, j) the name cij . 4. In each row, find the minimum value and write it Also, we introduce a variable xij that indicates whether down on the right hand side, below the negative sum there is a rook being placed on cell (i, j) or not: xi j = 1 figure. means “rook on (i, j)”, xi j = 0 means “no rook”. The objective function is the sum of all ci j , each value being selected 5. Add up alle row minima and the negative sum. by its “rook factor” of xi j . The feasibility constraint can be expressed as two sets of equalities: the sum over all x’s in This is a generalization of the simple bound of figure 10 a row or column, resp., must equal one. In mathematical in that we first choose the arbitrary numbers and add them language, we write down a model: to all cells in their respective columns. Then, like figure 10, we seek the minimum values in all respective rows and add them up, this time including the negative sum of the chosen minimize ∑ ∑ ci j xi j numbers. Setting these numbers all to zero would reproduce i j the calculation of figure 10. subject to (i) for all columns j ∑ xi j = 1 The calculation of lower bounds more than pays for iti self. The effect of strong lower bounds can be seen in fig(ii) for all rows i ure 12. The upper numbers in the vertices are the sums ∑ xi j = 1 j parval of the rook position values from figure 8, the lower numbers are lower bounds lbound for the sums of the miss(iii) xi j = 0 or 1. ing rooks. The total lower bounds are equal to their resp. sums of parval + lbound. These four model lines precisely capture the rooks problem, irrespective of the size of the grid or the values in the cells. Remember: the feasibility constraint determines 0 parval which objects are feasible to the problem, its expression are 10 lbound the equalities stated after the “subject to” marker. 3. In each column, add the assigned number to all the cell values. 9 7 8 2 10 6 10 8 12 16 13 9 5 4 8 4 6 4 9 6 Observation: if one removes (neglects) one or more of these equalities, there are less restrictions left to define feasibility. Thus, the set of all feasible solutions becomes larger, as we allow more objects into it. Consequently, the minimal objective value may at most decrease, this effect possibly arising from new objects with smaller objective values. 17 14 18 10 5 9 3 6 1 3 6 1 19 13 11 10 20 Figure 12: search tree with lower bounds In mathematical terminology, the removal of feasibility defining equalities (or inequalities) from a model is called a relaxation. Its effect is always that the optimum value of the relaxed problem is a lower bound to the desired optimum of the original problem. In our example, we could for instance relax all equalities (i). That would allow two or more rooks to sit in the same column while leaving intact the constraint “one per row”. In fact, this is the model equivalent of the simple lower bound from figure 10. In order to understand the lower bound property of the Now we only go down to an end vertex of the tree when there is a better solution than the current one. A significant Lower Bound Algorithm, consider the following slightly different model: reduction in solution enumeration. ! ∑ ∑ ci j xi j + ∑ λ j ∑ xi j − 1 minimize i subject to (i) j j i ∑ xi j = 1 for all columns j ∑ xi j = 1 for all rows i i (ii) The theory of the Bound Improvement Algorithm is beyond the scope of this paper. For an analysis and discussion see, for instance, [1]. Here, we give the most widely used formulation of it. The parameters are called λ j as above. All of them together form the vector λ. Bound Improvement Algorithm j (iii) xi j = 0 or 1 (iv) λ j arbitrary. 1. Set k = 0. Start with arbitrary values λ(0) , e.g. λ(0) = 0. It differs from the original rooks model only in the additional objective function term and the new arbitrary parameters λ j . Looking at the extra term more closely, it turns out to be meaningless: since we have constraint (i), it must always equal zero. So we see that, independent of the choice of the λ j , this second model has exactly the same set of feasible solutions and identical objective values for each solution. Thus, their respective optima are equal as well. The power of the extra objective term unfolds, when we now relax the constraint (i) as we did above. The optimum of the relaxed second model (the so-called Lagrangian relaxation of the original model) will now depend on the choice of the λ j . But since a relaxed optimum can at most reach the unrelaxed one from below, our relaxed second model must always deliver a lower bound to our unrelaxed original model. This completes the proof. For the sake of completeness, it remains to show that the algorithm really computes the optimum of the relaxed second model. This can be readily seen by rearranging its objective function: ! 2. Invoke the Lower Bound Algorithm with λ(k) , take back the bound and the numbers n j of row minima chosen in each resp. column j (cf. figure 11). 3. Define new vector λ(k+1) : let (k+1) λj (k) = λ j + sk (n j − 1) for all j. 4. Increase k by 1, repeat from step 2 as desired. The algorithm produces a sequence of lower bound values which is not necessarily monotonic, see figure 13. So one has to keep track of the best current bound value. Note that the number ∑ j n j of found column conflicts tends to decrease as the algorithm proceeds. lower bound number of conflicts best possible bound value ∑ ∑ ci j xi j + ∑ λ j ∑ xi j − 1 i = j j i ∑ ∑(ci j + λ j ) xi j − ∑ λ j . i j almost feasible solutions 200 iter. j The reader interested in the details of Lagrangian relaxation is referred to [1]. 2.6 Choice of the parameters Introducing the arbitrary numbers as parameters opens a vast degree of freedom in the calculation. As figure 11 demonstrates, it is possible in our example to choose the numbers in such a way that the lower bound equals the optimum. These are clearly the best bounds achievable, so the method is quite powerful. But how can we make this choice? The answer: there is no formula for the best values. In practice today, an algorithm named Subgradient Optimization is usually employed. It starts with arbitrary values and updates them iteratively. In every iteration, the entire lower bound algorithm has to be performed. This may be time comsuming, but is worth a certain number of iterations, as the lower bounds increase and save efforts in the search tree exploration process. Figure 13: typical bound improvement sequence The algorithm uses an auxiliary sequence sk which determines the length of the update step 3. Essentially any such sequence works if it obeys the following conditions (cf. [5]): • all sk are nonnegative, • sk converges to zero, • ∑k sk diverges to infinity. Examples for such sequences are Ck or √Ck where C is a positive constant. Several refinements to the algorithm have been suggested in the literature, for instance in [3, 2]. They have the potential to significantly reduce the number of iterations needed to reach a satisfactory bound. Yet all of them come at the expense of a rather tedious need to administer more free parameters. Adapting the Bound Improvement Algorithm to a specific problem is a substantial, yet worthwhile, subtask of the whole optimization project. 2.7 Upper Bounds According to the pruning criterion of section 2.3, we can best prune the search tree when we have strong lower bounds and/or good known feasible solutions. Their values are always upper bounds for the optimum, since the minimum objective value can be at most as high as that of any feasible solution. So how can we achieve strong upper bounds? Since any feasible solution can serve us, the way to go can be twofold. Either we produce many feasible solutions fast and choose their best value. Or we can spend some time on a more sophisticated way to construct a few “good” solutions. We will discuss both approaches. An algorithm that strives to compute a good, but not necessarily optimal, feasible solution is called a heuristic. Since feasible solutions to the rooks problem abound, there is a simple way of getting many of them fast: random choice. Place one rook randomly and reduce the grid by eliminating its row and column. Then do the same with the reduced grid until all rooks have been placed. This can be repeated several times in a row up to some prechosen time-out. In terms of the search tree, this method follows some randomly chosen path from the current vertex down to a solution. It ignores any information along the way, see figure 14. start vertex An interesting feature of the Bound Improvement Algorithm of section 2.6 is it that it produces a sequence of (mostly) infeasible solutions whose number of feasibility conflicts more or less decreases over the iterations, see figure 13. Remember: according to the relaxation we chose, we allowed objects as feasible solutions that have multiple rooks in their columns. If rooks are placed in the same column, this is called a conflict. If now the number of conflicts varies, we can pick one infeasible solution with the least number of conflicts from this sequence. If its number is zero, we even have a feasible solution ready at hand. If its number is positive, but not too high, we can feed the solution into a heuristic that attempts to remove these conflicts. A possible heuristic based on this idea could do the following. Conflict Removal Heuristic 1. Take solution with minimal number of conflicts from the Bound Improvement Algorithm. 2. Regard every rook in the solution as fixed when it stands alone in its column. Eliminate both its row and column from the grid. 3. Choose a column with more than one rook in it. For every rook in the column, seek the minimum cell value in its row without its current position in that row. Calculate the difference between this minimum and its current cell value. 4. From the rooks in the column, choose one with the largest difference to be the one assigned to that column. Place the others on their minimum value cells computed in step 3. 5. Go on with step 2 if there are still conflicts. Otherwise, output the found solution. The Conflict Removal Heuristic alternatingly reduces the problem size by accepting some rooks as being fixed, and resolves a conflict by deciding which one rook gets to stay in a column. The choice in step 4 is just one way of deciding, any other rule will also do. For an example of the Conflict Removal Heuristic, see figure 15. 9 13 20 16 18 22 25 Figure 14: random heuristic for solutions 2.8 Looking closer An example for a slightly more sophisticated heuristic is the following. Again we build up a solution rook by rook. We no longer choose both the row and column positions randomly, but only the row. Once a row is given, we seek the cell with minimum value and place the rook on it. After that, we again reduce the grid and iterate. 8 2 +6 fixed 4 4 differences 5 6 −2 6 3 +1 rook in solution next best position Figure 15: step 3 of the Conflict Removal Heuristic Note: a negative value difference may well occur, since bounds tend to be the lowest ones in the entire tree. Followthe Bound Improvement Algorithm uses modified cell val- ing them, we have a high chance of going straight into the ues for its internal rook placement. The Conflict Removal “general area” of the optimum. Heuristic only uses the given input cell values. This is illustrated in figure 17 where the tree vertices carry their total lower bound values. Following the minimal bound value of 10, we are directly led to the optimum. In 2.9 Applicability general, this is not as nicely the case. In large problem inAs easy as the design of a heuristic may seem, a word stances, the tree does in fact branch out somewhat, but does should be said about the applicability. Not all problems are so in a more or less “focused” way. This strategy of tree structured such that the feasibility constraint always allows exploration is called branch-and-bound. us to construct a feasible solution in a more or less arbitrary manner. This holds especially true when starting out with a partial solution. figure 16 shows an example of an enumeration tree in which not all subtrees contain feasible solutions. The problem setting is identical to the rooks problem, only the rooks have been replaced by queens. 10 19 13 10 18 000000 111111 111111 000000 000000 111111 000000 111111 000000 111111 000000 111111 000000 111111 000000 111111 000000 111111 000000 111111 000000 111111 000000 111111 14 111 000 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 11 00 00 11 00 11 000 111 00 11 000 111 000 111 000 111 000 111 000 111 000 111 11 00 00 11 00 11 000 111 00 11 000 111 000 111 000 111 000 111 000 111 000 111 11 00 00 11 00 11 00 11 00 11 11 00 00 11 00 11 00 11 00 00 11 11 11 00 00 11 00 00 11 11 00 11 00 11 00 00 11 11 00 11 00 11 00 11 11 00 00 11 00 11 00 11 00 11 00 11 00 11 11 00 00 11 00 11 00 11 00 11 00 11 00 11 11 00 11 11 00 00 11 00 11 11 00 00 11 00 11 00 11 00 11 00 11 00 11 00 11 14 10 111 000 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 000 111 11 00 00 11 00 11 00 11 00 11 00 11 00 11 00 11 10 00 11 11 00 00 11 00 11 10 00 11 11 00 00 11 00 11 00 11 11 00 00 11 00 11 Figure 16: search tree with queens In general, good knowledge about the problem structure is necessary for designing a successful heuristic. 3 Vertex Sequence There is another degree of freedom which can help us keep the search effort under control: the vertex sequence in which we build up the enumeration tree. The depth-first-search strategy we have used so far is canonical, yet there is a better one. We let the lower bounds be our guides. Whenever we have to choose another tree vertex to explore, we choose one with a minimal lower bound value among all vertices waiting to be explored. The rationale behind this is the following observation: consider the path from the tree root (the empty grid) down to the optimal solution. Every vertex on it carries a lower bound that cannot exceed the optimum objective value. But since the optimum is the lowest possible value, also these Figure 17: search tree with branch-and-bound strategy The branch-and-bound strategy possesses three important features: • It requires possibly huge amounts of storage for all the vertices that are candidates for exploration. • Its performance relies heavily on the quality of the lower bounds. If a problem does not offer the computation of strong bounds, this method may take prohibitive time (and storage) to proceed. • In choosing always a vertex with minimal lower bound, this bound is, in every iteration, a global lower bound for the problem instance. While the first two features rather represent limitations, the third one makes branch-and-bound the more desirable: the global lower bound steadily increases as the method proceeds. Using an upper bound heuristic (see section 2.7), one can commonly observe that from time to time better feasible solutions, i.e. global upper bounds, are found. So both global bounds converge monotonically towards each other, eventually meeting at the optimum value. A typical trajectory of a branch-and-bound run is shown in figure 18. objective value actual search optimality proof optimum found time Figure 18: a typical branch-and-bound trajectory One can see another typical feature of combinatorial optimization in general: the optimal solution is often found well before its optimality is proven by the lower bound. This means that branch-and-bound can be used as an improvement heuristic itself: just let it run until the gap between the two global bounds is as small as is acceptable, for instance maybe 10 per cent. Even with no more than the accepted gap as an optimality guarantee, there is a good chance that the optimum was already found. Strictly speaking, we have only introduced the bound step so far. The branch step is what we have silently used all the time: the local enumeration of all possibilities of placing a rook within a grid row. In terms of the tree, branching means the manner in which we branch out from a vertex. We will not go into discussion here, as this way is rather canonical for the rooks problem. 4 Summary of Combinatorial Optimization The typical setting of a combinatorial optimization problem is an abundance of feasible solutions and the task is to seek out one of them with minimum (or maximum) objective value. What makes such a problem hard is the fact that it is often not possible to actually compare the objective values of all feasible solutions. So special search techniques must be employed that mostly work with partial solutions in a clever way. 3. upper bound: a global value with the guarantee that it is no less than the optimum value. In particular, the objective value of any feasible solution is an upper bound. Upper bounds are strong when they are as low as possible. 4. vertex sequence: the sequence in which the tree vertices are explored. Branch-and-bound is one popular sequence, utilizing the lower bounds. The effects of introducing the ideas 2.–4. can be seen in figure 19 for the rooks problem. As the size of the grid increases up to 100, the different curves indicate the computation times to optimum. complete enumeration time [s] 100000 10000 1000 100 10 1 0.1 0.01 Branch-and-Bound respect best solution plus upper bounds plus lower bounds 20 40 100 grid size Figure 19: computation times over problem sizes Three main features clearly come out: • Complete enumeration without refinements is quickly out of practical reach as sizes grow. • The introduction of lower bounds is a major improvement that can possibly take out the strict (super-) exponential growth in time consumption. Thus, it is usually worth investing quite some time into the calculation of strong bounds. • The more sophisticated the search is, the more can the algorithm exploit stochastic luck. So a more or less accurate estimate of the remaining running time is almost never possible. 4.2 Linear and integer programming 4.1 The ingredients In the recent past, all-purpose LP/IP solvers have become In order to make the solution process work, four ingredients strong on the optimization market. In principle, these tools are able to solve combinatorial problems as well. Their adhave to be specified (here for a minimization problem): vantage clearly is that one purchases excellent know-how 1. branching strategy: the manner in which a tree ver- with their running licenses and updates, and the modeler is freed from thinking about the solution process. tex branches out upon exploration. Yet on the flip side, these software packages require that 2. lower bound: an estimate from below for the optimal the problem be modeled in their specific language which objective values of all feasible solutions in a specified may not be an easy task, depending on the problem. In subtree. Lower bounds are strong when they are as any case, it means one has to reorganize the given data into some specified input format. Also, due to their all-purpose high as possible. nature, commercial solvers cannot exploit special advantageous features in the problem at hand. A common other drawback is that they tend to consume huge amounts of storage for their execution, a possibly limiting factor for large problem instances. LP/IP software packages basically employ the same method ingredients as listed above. They merely use different techniques in filling in the details. Tailor-made combinatorial solvers can normally be restricted to much less memory consumption — possibly work directly on the data at hand — and can be written so as to exploit special problem features. So there is a certain trade-off between buying versus implementing a software solution in any case. The reader interested in linear/integer programming is referred to [4, 6] for an introduction. In the next section, we apply the optimization knowhow presented to the Disjoint Paths Problem. b a c e d f h g m i k a b c d e f g h i k m Figure 20: VPN solution in grid representation Figure 20 shows the feasible solution of figure 3 as a grid pattern: the marked entries in a row indicate the trunks that the row’s demand is routed over. Condition (a) is satWe now switch back to the VPN problem which was stated isfied, as one can see in the network. And, according to in section 1. How does it fit into the combinatorial opti- condition (b), there is no column with more than one entry mization framework? in it. This is the desired disjointness. (Note that, for clarFirst, we need to identify the feasible objects. In this ity of presentation, no cost values are given. They normally case, we expect as the output a routing of all given end-to- belong inside the grid, one per cell.) In order to compute an optimal solution, we need to end demand pairs of the VPN. A demand counts as routed when we exhibit a simple path through the network con- specify the four “ingredients” identified in section 4. necting its end nodes. Furthermore, the routing is feasible if no two of these paths share a common trunk. 5.1 Branching strategy Mathematically speaking, for each individual demand pair d we seek a subset T d of all network trunks. The feasi- From the rooks problem, we know: a vertex in the search tree always in principle looks like the entire problem. Placbility constraint then consists of two separate conditions: ing a rook implied eliminating its row and column from the grid, leaving a slightly smaller rooks problem. (a) the trunks in each set T d must constitute a simple path In the VPN case, there are always some demand pairs in the network connecting the end nodes of demand d, to be disjointly routed through some network. The only difference to the entire given problem instance is that not nec(b) no two sets may intersect: T d1 ∩ T d2 = ∅ for all de- essarily all or the same demands are still unrouted, and that mand pairs d1 , d2 . not necessarily all trunks of the network are still available. Several branching strategies are devisable. One strategy has proven to be especially well suited, see figure 21. It Second, we need to set up an objective function. We works as follows: are given a cost coefficient c on every trunk t, and this cost 5 Application to VPN planning t actually arises if one of the routing paths contains the resp. trunk: cost = ∑ ∑ ct . d t∈T d The VPN problem can be visualized so that it looks similar to the rooks problem of section 2, see figure 20. The rows of the grid correspond to the VPN demands, and the columns correspond to the trunks of the network. All cells in one and the same column carry identical cost values, as it does not matter which demand path contains the column’s trunk. Branching Strategy 1. Choose an unrouted demand d = (mn) and one of its end nodes n according to some rule (maybe randomly). 2. For all trunks t = (np) connected to node n, do: 3. reserve trunk t for the routing of demand d (i.e., put t into the set T d ) 4. form a new subproblem on the network less trunk t and with demand d altered to d = (mp). If m = p, remove demand d from the list of unrouted demands. m m t n m p p The Bound Improvement Algorithm (cf. section 2.6) is readily applicable as well. Just two slight differences are to be observed: • A restriction applies to step 3: the λ-values must stay nonnegative. This can be achieved by setting them to zero if the update formula results in a negative value. (The reason for this is that condition (b) restricts the number of paths containing any one trunk to at most one, not to exactly one, as in the rooks problem.) m • The numbers n j in the rooks problem are replaced by numbers nt that for each trunk t in the network count the number of routing paths containing it. t t p Figure 21: branching strategy 5.2 Lower bounds The minimum cost paths needed in step 2 are to be computed by the well-known Dijkstra Algorithm. Presenting it is beyond the scope of this paper, for an introduction see, for instance, [1]. 5.3 Upper bounds Calculation of lower bounds works here almost the same way as in the rooks problem, see also figure 22: Lower Bound Algorithm 1. Assign to each column t (in the grid of figure 20) an arbitrary nonnegative parameter value λt . 2. Now ignore the disjointness condition (b) and compute for every row a minimum cost path connecting the end nodes of the row’s demand pair. 3. Add up the costs of all minimum cost paths and subtract the sum of the parameter values from it. Unfortunately, the VPN problem has the property indicated in section 2.9, namely that in general not all subtrees of the entire enumeration tree contain feasible solutions. So a heuristic may or may not be successful, depending on the tree vertex it is run in. With this caveat, the following heuristic has proven to be very useful. It is a straightforward analogue of the algorithm given in section 2.8. A conflict is a trunk which is contained in more than one routing path. Conflict Removal Heuristic 1. Take solution with minimal number of conflicts from the Bound Improvement Algorithm. 2. Regard every routing path in the solution as fixed when none of its trunks is a conflict. Eliminate all its trunks from the network. 3. Choose a conflict trunk t. For every demand whose routing path contains t, compute the minimum cost path in the network without t. Calculate the difference between its cost and the cost of the original path. λa λb λc ≥ 0 ... λm − ∑t λt min min (nt = 0 2 1 1 0 0 1 1 0 0 2) Figure 22: lower bounds 4. From the demands in step 3, choose one with the largest cost difference to be the one allowed to “occupy” trunk t. Replace the other paths by their resp. minimum cost paths computed in step 3. 5. Go on with step 2 if there are still conflicts. Otherwise output the found solution. min 5.4 Vertex sequence bound Here the branch-and-bound sequence is appropriate. Note that the subproblems generated upon vertex exploration (see figure 21) are each specified by a subnetwork of the given network and by demands different from the original ones given. So every vertex kept track of necessitates a consider- References able storage amount. A careful design of the data structures [1] R. K. Ahuja, T. L. Magnanti, and J. B. Orlin: Network used may reduce memory consumption significantly. Flows, Prentice Hall, Englewood Cliffs (1993). 6 Conclusion [2] M. S. Bazaraa and J. J. Goode: A Survey of Various Tactics for Generating Lagrangian Multipliers in the Context of Lagrangian Duality, in European Journal of Operational Research 3, pp. 322–328. [3] M. Held, P. Wolfe, and H. Crowder: Validation of The techniques of Combinatorial Optimization offer a Subgradient Optimization, in Mathematical Programbroad tool kit that is applicable to a vast variety of pracming 6 (1974), pp. 62–88. tical problems. Optimizing over discrete objects has less of a straightforward calculation, but more of a search pro- [4] G. L. Nemhauser and L. A. Wolsey: Integer and Comcess. We have presented and exemplified the ingredients of binatorial Optimization, John Wiley and Sons, New a typical such process. York (1988). The solution of the stated VPN routing problem has been a successful project within T-Systems. Its outcome was the efficient implementation of all the algorithms presented, plus several more refinements. Reliable VPN routing at minimum cost is now a service product available from T-Systems, Dept. E46, Center of Technology, Darmstadt, Germany. [5] B. T. Polyak: A General Method for Solving Extremum Problems, in Soviet Mathematics Doklady 8 (1967), pp. 593–597. [6] H. M. Salkin and K. Mathur: Foundations of Integer Programming, Elsevier Science Publishing, New York (1989).