1. No category

The CVX User’s Guide
Michael C. Grant and Stephen P. Boyd
CVX Research, Inc.
1
Contents
 Introduction
 Installation
 A Quick Start – Demo
 The Basics
 The DCP Ruleset
 Semidefinite Programming Mode
 Geometric Programming Mode
 Examples
2
Introduction
 What is CVX ?
CVX is a modeling system for constructing and solving disciplined
convex programs (DCPs).
CVX supports a number of standard problem types, including linear
and quadratic programs (LPs/QPs), second-order cone programs
(SOCPs), and semidefinite programs (SDPs).
CVX is implemented in Matlab, effectively turning Matlab into an
optimization modeling language.
 Website: http://cvxr.com/cvx/
3
Introduction
 What is disciplined convex programming ?
Disciplined convex programming is a methodology for constructing
convex optimization problems proposed by Michael Grant, Stephen
Boyd, and Yinyu Ye.
It is meant to support the formulation and construction of
optimization problems that the user intends from the outset to be
convex.
Disciplined convex programming imposes a set of conventions or
rules, which we call the DCP ruleset.
Problems that violate the ruleset are rejected : even when the problem is
convex.
That is not to say that such problems cannot be solved using DCP; they
just need to be rewritten in a way that conforms to the DCP ruleset.
4
Installation
 Supported platforms
CVX is supported on 32-bit and 64-bit versions of Linux, Mac OSX,
and Windows.
32-bit platforms: MATLAB version 7.5 (R2007b) or later is required.
64-bit platforms: MATLAB version 7.8 (R2009a) or later is required.
 Setup:
Download the latest version of CVX.
http://cvxr.com/cvx/download/
Unpack the file anywhere you like.
Ex) C:\personal\cvx
Change directories to the top of the CVX distribution and run the
cvx_setup command.
5
Installation
 Example:
% example_CVX.m:
close all;
clear all;
clc
addpath('cvx');
cvx_setup
6
A Quick Start – Demo
 To delineate CVX specifications from surrounding Matlab code,
they are preceded with the statement cvx_begin and followed
with the statement cvx_end.
 A specification can include any ordinary Matlab statements, as
well as special CVX-specific commands for declaring primal and
dual optimization variables and specifying constraints and
objective functions.
 Example script: quickstart
~\cvx\examples\quickstart
7
A Quick Start – Demo
 Least-squares problem:
where
(
and
and
is a skinny full rank matrix
).
The least-squares solution:
Using Matlab
Using CVX, the problem can be solved as follows:
cvx_begin
variable x(n)
minimize( norm(A*x-b) )
cvx_end
declares x to be an optimization variable of
dimension n.
specifies the objective function to be
minimized.
8
A Quick Start – Demo
 Least-squares problem – example code
%
m
A
b
Input data
= 4; n = 3;
= randn(m,n);
= randn(m,1);
% Matlab version
x_ls = A \ b;
% cvx version
cvx_begin
variable x(n)
minimize( norm(A*x-b) )
cvx_end
Randomly generate a 4x3 matrix A, a 4x1 vector b
Matlab code for the least-squares solution
CVX code
% Compare
disp( ‘Results:');
disp( sprintf( ‘\nnorm(A*x_ls-b): %6.4f\n', norm(A*x_ls-b)));
disp( sprintf( 'norm(A*x-b):
%6.4f\n', norm(A*x-b)));
disp( sprintf( 'cvx_optval:
%6.4f\n', cvx_optval));
disp( sprintf( 'cvx_status:
%s\n', cvx_status));
disp( 'Verify that x_ls == x:' );
disp( [ '
x_ls = [ ', sprintf( '%7.4f ', x_ls ), ']' ] );
disp( [ '
x
= [ ', sprintf( '%7.4f ', x ), ']' ] );
9
A Quick Start – Demo
 Least-squares problem – results
Results:
norm(A*x_ls-b): 0.5165 Solved by the least-squares,
norm(A*x-b): 0.5165
Solved by CVX, is the same as the above least-squares solution !
cvx_optval:
0.5165
cvx_optval contains the value of the objective function
cvx_status:
Solved
cvx_status contains a string describing the status of the calculation
Verify that x_ls == x:
x_ls = [ -0.4209 -0.5287 0.0459 ]
x = [ -0.4209 -0.5287 0.0459 ]
10
A Quick Start – Demo
 Least-squares problem – error message
If the objective function is replaced with
% cvx version
cvx_begin
variable x(n)
maximize( norm(A*x-b) )
cvx_end
You will get an error message stating that a convex function cannot
be maximized (at least in disciplined convex programming):
??? Error using ==> maximize
Disciplined convex programming error:
Objective function in a maximization must be concave.
11
A Quick Start – Demo
 Bound-constrained least squares:
where l and u are given data vectors with the same dimension as x.
If you have the Matlab Optimization Toolbox, you can use quadprog
to solve the problem as follows:
x_qp = quadprog( 2*A’*A, -2*A’*b, [], [], [], [], l, u );
Using CVX, the problem can be solved as follows:
cvx_begin
variable x(n)
minimize( norm(A*x-b) )
subject to
l <= x <= u
cvx_end
12
A Quick Start – Demo
 Bound-constrained least squares – example code
% input data
m=4; n=3;
A = randn(m,n);
b = randn(m,1);
Input data:
4x3 matrix A
4x1 vector b
3x1 vectors l and u
bnds = randn(n,2);
l = min( bnds, [] ,2 );
u = max( bnds, [], 2 );
% Quadprog version
x_qp = quadprog( 2*A'*A, -2*A'*b, [], [], [], [], l, u );
% cvx version
cvx_begin
variable x(n)
minimize( norm(A*x-b) )
subject to
l <= x <= u
cvx_end
% Compare
disp( 'Verify that l <= x_qp == x
disp( [ '
l
= [ ', sprintf(
disp( [ '
x_qp = [ ', sprintf(
disp( [ '
x
= [ ', sprintf(
disp( [ '
u
= [ ', sprintf(
Matlab Toolbox
CVX code
<= u:'
'%7.4f
'%7.4f
'%7.4f
'%7.4f
);
',
',
',
',
l ),
x_qp
x ),
u ),
']' ] );
), ']' ] );
']' ] );
']' ] );
13
A Quick Start – Demo
 Bound-constrained least squares – results
Verify that l <= x_qp == x <= u:
l
= [ -1.0078 -0.7420 0.0880 ]
The lower bound of x
x_qp = [ -0.1315 -0.0459 0.6207 ] Solved by Matlab Toolbox
x
= [ -0.1315 -0.0459 0.6207 ]
Solved by CVX
u
= [ -0.1315 0.3899 1.0823 ]
The upper bound of x
14
A Quick Start – Demo
 Other norms and functions- Chebyshev norm
Using Matlab Optimization Toolbox, you can use linprog to solve the
problem as follows:
f = [ zeros(n,1); 1 ];
Ane = [ +A, -ones(m,1); ...
-A, -ones(m,1) ];
bne = [ +b; -b ];
xt = linprog(f,Ane,bne);
x_cheb = xt(1:n,:);
Using CVX, the problem can be solved as follows:
cvx_begin
variable x(n)
minimize( norm(A*x-b,Inf) )
cvx_end
15
A Quick Start – Demo
 Other norms and functions- Chebyshev norm
% input data
m=4; n=3;
A = randn(m,n);
b = randn(m,1);
% linprog version
f
= [ zeros(n,1); 1
Ane = [ +A,
-ones(m,1)
-A,
-ones(m,1)
bne = [ +b;
-b
xt
= linprog(f,Ane,bne);
x_lp = xt(1:n,:);
% cvx version
cvx_begin
variable x(n)
minimize( norm(A*x-b,Inf) )
cvx_end
Input data:
4x3 matrix A
4x1 vector b
];
; ...
];
];
Matlab Toolbox
CVX code
% Compare
disp( 'Verify that x_lp == x:' );
disp( [ '
x_lp = [ ', sprintf( '%7.4f ', x_lp ), ']' ] );
disp( [ '
x
= [ ', sprintf( '%7.4f ', x ), ']' ] );
16
A Quick Start – Demo
 Other norms and functions-
norm
Using Matlab Optimization Toolbox, you can use linprog to solve the
problem as follows:
f = [ zeros(n,1); ones(m,1); ones(m,1) ];
Aeq = [ A, -eye(m), +eye(m) ];
lb = [ -Inf(n,1); zeros(m,1); zeros(m,1) ];
xzz = linprog(f,[],[],Aeq,b,lb,[]);
x_l1 = xzz(1:n,:) - xzz(n+1:end,:);
Using CVX, the problem can be solved as follows:
cvx_begin
variable x(n)
minimize( norm(A*x-b,1) )
cvx_end
17
A Quick Start – Demo
 Other norms and functions% input data
m=4; n=3;
A = randn(m,n);
b = randn(m,1);
norm
Input data:
4x3 matrix A
4x1 vector b
% Matlab version
f
= [ zeros(n,1); ones(m,1); ones(m,1) ];
Aeq = [ A,
-eye(m),
+eye(m)
];
lb
= [ -Inf*ones(n,1); zeros(m,1); zeros(m,1) ];
xzz = linprog(f,[],[], Aeq,b,lb,[]);
x_lp = xzz(1:n,:) - xzz(n+1:2*n,:);
% cvx version
cvx_begin
variable x(n)
minimize( norm(A*x-b,1) )
cvx_end
Matlab Toolbox
CVX code
% Compare
disp( 'Verify that x_lp == x:' );
disp( [ '
x_lp = [ ', sprintf( '%7.4f ', x_lp ), ']' ] );
disp( [ '
x
= [ ', sprintf( '%7.4f ', x ), ']' ] );
18
A Quick Start – Demo
 Other norms and functions – largest k-norm
where
denotes the i-th largest element of the absolute
values of the entries of
.
k=1 
norm, k=m 
norm.
Consider the problem
Using CVX, the problem can be solved as follows:
k = 5;
cvx_begin
variable x(n);
minimize( norm_largest(A*x-b,k) );
cvx_end
19
A Quick Start – Demo
 Other norms and functions – largest k-norm
% input data
m=16; n=8;
A = randn(m,n);
b = randn(m,1);
% cvx specification
k = 5;
cvx_begin
variable x(n)
minimize( norm_largest(A*x-b,k) )
cvx_end
Input data:
16x8 matrix A
16x1 vector b
CVX code
% Compare
temp = sort(abs(A*x-b));
disp( sprintf( '\nResults:\n--------\nnorm_largest(A*xb,k): %6.4f\ncvx_optval: %6.4f\ncvx_status: %s\n', norm_largest(A*x-b,k), cvx_optval,
cvx_status ) );
disp( 'Optimal vector:' );
disp( [ '
x
= [ ', sprintf( '%7.4f ', x ), ']' ] );
disp( sprintf( 'Residual vector; verify a tie for %d-th place (%7.4f):', k, temp(end-k+1) ) );
disp( [ '
A*x-b = [ ', sprintf( '%7.4f ', A*x-b ), ']' ] );
disp( ' ' );
20
A Quick Start – Demo
 Other norms and functions – Huber penalty minimization problem
Huber penalty function:
The Huber penalty function is convex, and has been provided in the CVX
function library.
cvx_begin
variable x(n);
minimize( sum(huber(A*x-b)) );
cvx_end
21
A Quick Start – Demo
 Other norms and functions – Huber penalty minimization problem
% input data
m=16; n=8;
A = randn(m,n);
b = randn(m,1);
% cvx specification
cvx_begin
variable x(n)
minimize( sum(huber(A*x-b)) )
cvx_end
Input data:
16x8 matrix A
16x1 vector b
CVX code
% Compare
disp( sprintf( '\nResults:\n--------\nsum(huber(A*xb)): %6.4f\ncvx_optval: %6.4f\ncvx_status: %s\n', sum(huber(A*x-b)), cvx_optval, cvx_status ) );
disp( 'Optimal vector:' );
disp( [ '
x
= [ ', sprintf( '%7.4f ', x ), ']' ] );
disp( 'Residual vector:' );
disp( [ '
A*x-b = [ ', sprintf( '%7.4f ', A*x-b ), ']' ] );
disp( ' ' );
22
A Quick Start – Demo
 Other constraints
min ||𝐀𝐀𝐀𝐀 − 𝐛𝐛||2
𝐱𝐱
subject to 𝐂𝐂𝐂𝐂 = 𝐝𝐝
||𝐱𝐱||∞ ≤ 1
Both of the added constraints conform to the DCP rules, and so are
accepted by CVX.
After the cvx_end command, CVX converts this problem to an SOCP,
and solves it.
cvx_begin
variable x(n);
minimize( norm(A*x-b) );
subject to
C*x == d;
norm(x,Inf) <= 1;
cvx_end
23
A Quick Start – Demo
 Other constraints – example code
% input data
m=16; n=8;
A = randn(m,n);
b = randn(m,1);
%
p
C
d
More input data
= 4;
= randn(p,n);
= randn(p,1);
% cvx specification
cvx_begin
variable x(n);
minimize( norm(A*x-b) )
subject to
C*x == d
norm(x,Inf) <= 1
cvx_end
% Compare
disp( 'Optimal vector:' );
disp( [ '
x
= [ ', sprintf(
disp( 'Residual vector:' );
disp( [ '
A*x-b = [ ', sprintf(
disp( 'Equality constraints:' );
disp( [ '
C*x
= [ ', sprintf(
disp( [ '
d
= [ ', sprintf(
Input data
CVX code
'%7.4f ', x ), ']' ] );
'%7.4f ', A*x-b ), ']' ] );
'%7.4f ', C*x ), ']' ] );
'%7.4f ', d
), ']' ] );
24
A Quick Start – Demo
 An optimal trade-off curve
The following code solves the problem of minimizing
for a logarithmically spaced vector of (positive) values of
Traditional Matlab code and CVX specifications can be mixed to form
and solve the multiple optimization problems.
This gives us points on the optimal trade-off curve between
and
.
25
A Quick Start – Demo
 An optimal trade-off curve – example code
% input data
m=16; n=8;
A = randn(m,n);
b = randn(m,1);
%
disp( ' ' );
disp( 'Generating tradeoff curve...' );
cvx_pause(false);
gamma = logspace( -2, 2, 20 );
l2norm = zeros(size(gamma));
l1norm = zeros(size(gamma));
fprintf( 1, '
gamma
norm(x,1)
norm(A*x-b)\n' );
fprintf( 1, '---------------------------------------\n' );
for k = 1:length(gamma),
fprintf( 1, '%8.4e', gamma(k) );
cvx_begin quiet
variable x(n)
minimize( norm(A*x-b)+gamma(k)*norm(x,1) )
cvx_end
l1norm(k) = norm(x,1);
l2norm(k) = norm(A*x-b);
fprintf( 1, '
%8.4e
%8.4e\n', l1norm(k), l2norm(k) );
end
figure; plot( l1norm, l2norm ); xlabel( 'norm(x,1)' ); ylabel( 'norm(A*x-b)' ); grid on;
CVX code
26
A Quick Start – Demo
 An optimal trade-off curve – example code
4
3.8
norm(A*x-b)
3.6
3.4
3.2
3
2.8
2.6
0
High
0.2
0.4
0.6
0.8
1.2
1
norm(x,1)
1.4
1.6
1.8
2
Low
27
The Basics
 The command: cvx_begin and cvx_end
All CVX models must be preceded by the command cvx_begin and
terminated with the command cvx_end.
All variable declarations, objective functions, and constraints should
fall in between.
The cvx_begin command may include one more more modifiers:
cvx_begin quiet
• Prevents the model from producing any screen output while it is being solved.
cvx_begin sdp
• Semidefinite programming mode.
cvx_begin gp
• Geometric programming mode.
These modifiers may be combined when appropriate
Ex) cvx_begin sdp quiet
• SDP mode and silences the solver output.
28
The Basics
 Variables
All variables must be declared using the variable command (or
variables command) before they can be used in constraints or an
objective function.
A variable command includes the name of the variable, an optional
dimension list, and one or more keywords that provide additional
information about the content or structure of the variable.
Variables can be real or complex scalars, vectors, matrices, or n dimensional arrays.
variable X
variable Y(20,10)
variable Z(5,5,5)
These declares a total of 326 (scalar) variables: a scalar X, a 20x10 matrix
Y (containing 200 scalar variables), and a 5x5x5 array Z (containing 125
scalar variables).
29
The Basics
 Variables
Variable declarations can also include one or more keywords to
denote various structures or conditions on the variable.
variable w(50) complex: a complex variable
variable p(10) integer: integer variables
variable q binary: a binary variable
A variety of keywords are available to help construct variables with
matrix structure such as symmetry or bandedness.
variable Y(50,50) symmetric: a real 50 × 50 symmetric matrix variable
variable Z(100,100) hermitian toeplitz: a 100 × 100 Hermitian Toeplitz
matrix variable
30
The Basics
 Variables
The currently supported structure keywords are:
banded(lb,ub)
skew_symmetric
lower_bidiagonal
upper_bidiagonal
diagonal
symmetric
lower_hessenberg
upper_hankel
hankel
toeplitz
lower_triangular
upper_hessenberg
hermitian
tridiagonal
upper_triangular
The variables statement is provided which allows you to declare
multiple variables.
variables x1 x2 x3 y1(10) y2(10,10,10) ;
31
The Basics
 Objective functions
Declaring an objective function requires the use of the minimize or
maximize function, as appropriate.
The objective function in a call to minimize must be convex; the
objective function in a call to maximize must be concave.
minimize( norm( x, 1 ) )
maximize( geo_mean( x ) )
At most one objective function may be declared in a CVX
specification, and it must have a scalar value.
If no objective function is specified, the problem is interpreted as a
feasibility problem, which is the same as performing a minimization with
the objective function set to zero.
In this case, cvx_optval is either 0, if a feasible point is found, or +Inf, if
the constraints are not feasible.
32
The Basics
 Constraints
The following constraint types are supported in CVX:
Equality == constraints, where both the left- and right-hand sides are
affine expressions.
Less-than <= inequality constraints, where the left-hand expression is
convex, and the right-hand expression is concave.
Greater-than >= constraints, where the left-hand expression is concave,
and the right-hand expression is convex.
The non-equality operator ~= may never be used in a constraint; in
any case, such constraints are rarely convex.
33
The Basics
 Functions
The base CVX function library includes a variety of convex, concave,
and affine functions which accept CVX variables or expressions as
arguments.
Many are common Matlab functions such as sum, trace, diag, sqrt,
max, and min, re-implemented as needed to support CVX
Others are new functions not found in Matlab.
A complete list of the functions in the base library can be found in
Reference guide.
It is also possible to add your own new functions; see Adding new
functions to the atom library.
34
The Basics
 Set membership
CVX supports the definition and use of convex sets.
The base library includes the cone of positive semidefinite n × n
matrices, the second-order or Lorentz cone, and various norm balls.
A complete list of sets supplied in the base library is given in
Reference guide.
To require that the matrix expression X be symmetric positive
semidefinite ( ), we use the syntax
X == semidefinite(n)
35
The Basics
 Set membership
If this use of equality constraints to represent set membership
remains confusing or simply aesthetically displeasing, we have
created a “pseudo-operator” <In> that you can use in its place.
The semidefinite constraint above can be replaced by
X <In> semidefinite(n);
This is exactly equivalent to using the equality constraint operator, but if
you find it more pleasing, feel free to use it.
Implementing this operator required some Matlab trickery, so don’t
expect to be able to use it outside of CVX models.
36
The Basics
 Set membership
Sets can be combined in affine expressions, and we can constrain an
affine expression to be in a convex set.
A*X*A’-X <In> B*semidefinite(n)*B’;
where X is an n × n symmetric variable matrix, and A and B are n × n
constant matrices.
This constraint requires that
, for some
.
37
The Basics
 Set membership
CVX also supports sets whose elements are ordered lists of
quantities.
Consider the secondorder or Lorentz cone,
where epi denotes the epigraph of a function.
An element of
is an ordered list, with two elements:
• The first is an m-vector, and the second is a scalar.
We can use this cone to express the simple least-squares problem from
the section Least squares (in a fairly complicated way) as follows:
38
The Basics
 Set membership
CVX uses Matlab’s cell array facility to mimic this notation:
cvx_begin
variables x(n) y;
minimize( y );
subject to
{ A*x-b, y } <In> lorentz(m);
cvx_end
The function call lorentz(m) returns an unnamed variable (i.e., a pair
consisting of a vector and a scalar variable), constrained to lie in the
Lorentz cone of length m.
So the constraint in this specification requires that the pair { A*x-b,
y } lies in the appropriately-sized Lorentz cone.
39
The Basics
 Dual variables
When a disciplined convex program is solved, the associated dual
problem is also solved.
The optimal dual variables, each of which is associated with a
constraint in the original problem, give valuable information about
the original problem, such as the sensitivities with respect to
perturbing the constraints.
To get access to the optimal dual variables in CVX, you simply
declare them, and associate them with the constraints.
40
The Basics
 Dual variables
Consider, for example, the LP
with variable
, and m inequality constraints.
To associate the dual variable y with the inequality constraint
this LP, we use the following syntax:
n = size(A,2);
cvx_begin
variable x(n);
dual variable y;
minimize( c’ * x );
subject to
y : A * x <= b;
cvx_end
in
After the cvx_end statement is processed, and assuming the optimization
was successful, CVX assigns numerical values to x and y - the optimal
primal and dual variable values, respectively.
41
The Basics
 Assignment and expression holders
Anyone with experience with C or Matlab understands the difference
between the single-equal assignment operator = and the doubleequal equality operator ==.
This distinction is vitally important in CVX as well, and CVX takes
steps to ensure that assignments are not used improperly.
42
The DCP Ruleset
 The DCP Ruleset
CVX enforces the conventions dictated by the disciplined convex
programming ruleset, or DCP ruleset for short.
CVX will issue an error message whenever it encounters a violation
of any of the rules, so it is important to understand them before
beginning to build models.
The rules are drawn from basic principles of convex analysis, and
are easy to learn, once you’ve had an exposure to convex analysis
and convex optimization.
The DCP ruleset is a set of sufficient, but not necessary, conditions
for convexity.
So it is possible to construct expressions that violate the ruleset but
are in fact convex.
If a convex (or concave) function is not recognized as convex or
concave by CVX, it can be added as a new atom; see Adding new
functions to the atom library.
43
The DCP Ruleset
 The DCP Ruleset
Ex)
The entropy function
- sum( x .* log( x ) )  X
sum( entr( x ) )  O
The function
CVX will reject it, because its
concavity does not follow from any
of the composition rules.
which is in the base CVX library, and thus recognized
as concave by CVX.
norm( [ x 1 ] )  O
sqrt( x^2 + 1 )  X
44
The DCP Ruleset
 A taxonomy of curvature
In disciplined convex programming, a scalar expression is classified
by its curvature.
There are four categories of curvature: constant, affine, convex, and
concave.
For a function f : Rn  R defined on all Rn, the categories have the
following meanings:
Of course, there is significant overlap in these categories.
For example, constant expressions are also affine, and (real) affine
expressions are both convex and concave.
45
The DCP Ruleset
 Top-level rules
CVX supports three different types of disciplined convex programs:
A minimization problem, consisting of a convex objective function and
zero or more constraints.
A maximization problem, consisting of a concave objective function and
zero or more constraints.
A feasibility problem, consisting of one or more constraints and no
objective.
 Constraints
Three types of constraints may be specified in disciplined convex
programs:
An equality constraint, constructed using ==, where both sides are affine.
A less-than inequality constraint, using <=, where the left side is convex
and the right side is concave.
A greater-than inequality constraint, using >=, where the left side is
concave and the right side is convex.
46
The DCP Ruleset
 Constraints
One or both sides of an equality constraint may be complex;
inequality constraints, on the other hand, must be real.
A complex equality constraint is equivalent to two real equality
constraints, one for the real part and one for the imaginary part.
An equality constraint with a real side and a complex side has the
effect of constraining the imaginary part of the complex side to be
zero.
Strict inequalities <, > are interpreted in an identical fashion to
nonstrict inequalities >=, <=.
47
Semidefinite Programming Mode
 Semidefinite programming mode
Those who are familiar with semidefinite programming (SDP) know
that the constraints that utilize the set semidefinite(n) typically
expressed using linear matrix inequality (LMI) notation.
For example, given X = XT
Rn×n, the constraint X
X
Sn+; that is, that X is positive semidefinite.
0 denotes that
CVX provides a special SDP mode that allows this LMI notation to be
employed inside CVX models using Matlab’s standard inequality
operators >=, <=.
In order to use it, one simply begins a model with the statement
cvx_begin sdp or cvx_begin SDP instead of simply cvx_begin.
48
Semidefinite Programming Mode
 Semidefinite programming mode
When SDP mode is engaged, CVX interprets certain inequality
constraints in a different manner. To be specific:
Inequality constraints involving real, square matrices are interpreted as
follows:
If either side is complex, then the inequalities are interpreted as follows:
49
Semidefinite Programming Mode
 Semidefinite programming mode
Ex) the CVX model
cvx_begin
variable Z(n,n) hermitian toeplitz
dual variable Q
minimize( norm( Z - P, ’fro’ ) )
Z == hermitian_semidefinite( n ) : Q;
cvx_end
can also be written as follows:
cvx_begin sdp
variable Z(n,n) hermitian toeplitz
dual variable Q
minimize( norm( Z - P, ’fro’ ) )
Z >= 0 : Q;
cvx_end
50
Geometric Programming Mode
 Geometric programming mode
Geometric programs (GPs) are special mathematical programs that
can be converted to convex form using a change of variables.
The convex form of GPs can be expressed as DCPs, but CVX also
provides a special mode that allows a GP to be specified in its native
form.
CVX will automatically perform the necessary conversion, compute a
numerical solution, and translate the results back to the original
problem.
To utilize GP mode, you must begin your CVX specification with the
command cvx_begin gp or cvx_begin GP instead of simply cvx_begin.
51
Geometric Programming Mode
 Geometric programming mode
For example, the following code, found in the example library at
gp/max_volume_box.m, determines the maximum volume box subject
to various area and ratio constraints:
cvx_begin gp
variables w h d
maximize( w * h * d )
subject to
2*(h*w+h*d) <= Awall;
w*d <= Afloor;
alpha <= h/w >= beta;
gamma <= d/w <= delta;
cvx_end
52
Geometric Programming Mode
 Top-level rules
CVX supports three types of geometric programs:
A minimization problem, consisting of a generalized posynomial objective
and zero or more constraints.
A maximization problem, consisting of a monomial objective and zero or
more constraints.
A feasibility problem, consisting of one or more constraints.
 Contraints
Three types of constraints may be specified in geometric programs:
An equality constraint, constructed using ==, where both sides are
monomials.
A less-than inequality constraint <= where the left side is a generalized
posynomial and the right side is a monomial.
A greater-than inequality constraint >= where the left side is a monomial
and the right side is a generalized posynomial.
53
Examples – Spectrum Sharing Problem
 Spectrum Sharing in Cognitive Radio Networks
Consider a single channel transmission
Narrow-band transmission with deterministic channels
The secondary transmission can be represented by
=
y (n) Hx(n) + z (n)
H ∈  Nr × Nt denotes the secondary users’s channel
where N t and N r are the number of antennas at the secondary
transmitter and receiver, repectively.
y (n) and x(n) are the received and transmitted signal vector,
repectively, and n is the symbol index (or time index).
z (n) is the additive noise vector at the secondary receiver,
z (n) ~ CN (0, I ) .
Let the transmit covariance matrix (spatial spectrum) of the secondary
H
user be denoted by S , S = Ε[ x( n) x ( n)] .
54
Examples – Spectrum Sharing Problem
 Spectrum Sharing in Cognitive Radio Networks
Consider a single channel transmission
It is assumed that the ideal Gaussian code-book with infinitely large
number of codeword symbols is used, x( n) ~ CN (0, S) .
The transmit covariance matrix S can be further represented by its
eigenvalue decomposition expressed as
N ×d
S = VPV H
V H V = I , contains the eigenvectors of S , and is
where V ∈  t ,
also termed as the precoding matrix.
• Each column of V is the precoding vector for one transmitted data stream.
• d is usually reffered to as the degree of spatial multiplexing.
is a d × d diagonal matrix, and its diagonal elements are the positive
eigenvalues of S , and also represent the assigned transmit powers for
their corresponding data streams.
If d = 1, the corresponding transmission strategy is termed as
beamforming while in the case of d > 1, it is termed as spatial multiplexing.
P
55
Examples – Spectrum Sharing Problem
 Spectrum Sharing in Cognitive Radio Networks
System model
M 1 antennas
…
…
…
Tr (G k SG kH )
GK
N t antennas
N r antennas
…
…
Secondary Transmitter
Interference power to the k-th primary user
Primary Receiver K
Primary Receiver 1
G1
M K antennas
H
Transmit rate of the secondary user
log 2 | I + HSH H |
Secondary Receiver
* | X | denotes the determinant.
56
Examples – Spectrum Sharing Problem
 Spectrum Sharing in Cognitive Radio Networks
The optimization problem for the optimal S
max log 2 | I + HSH |
H
subject to Tr (S) ≤ Pt
Transmit power constraint
Tr (G k SG kH ) ≤ Γ k ,
S
0
k =1,..., K
Interference power constraint
The spatial spectrum S must be positive semi-definite
The above problem maximizes the secondary user’s channel
capacity assuming that the secondary user’s channel is known
perfectly at the secondary receiver.
This problem is a convex optimization problem.
The objective function is a concave function of S.
All of the constraints specify a convex set of S.
It can be solved by CVX.
57
Examples – Spectrum Sharing Problem
 Spectrum Sharing in Cognitive Radio Networks
CVX Code:
[Nr Nt]=size(H);
ID=eye(Nr,Nr);
cvx_begin sdp quiet
variable S(Nt,Nt) hermitian semidefinite
maximize( log_det(ID + H*S*H') )
subject to
real(trace(S)) <= Pt
real(trace(G*S*G')) <= gamma
cvx_end
58
Examples – Spectrum Sharing Problem
 Spectrum Sharing in Cognitive Radio Networks
Results:
K=1,
Γ=0.5, M=1, Nt=4, Nr=4
Achievable Sum-rate
Interference to Primary user
20
5
18
4.5
4
16
3.5
Interference
Sum Rate
14
12
10
8
3
2.5
2
1.5
6
1
4
2
-5
Γ
0.5
0
5
SNR-dB
10
15
0
-5
0
5
SNR-dB
10
15
59
Examples – Power Allocation Problem
 Two-way relay system
Enhance spectral efficiency
Reduce time slots for information exchange between S1 and S2 via R
Transmission based on analogue network coding
1st time slot: S1 and S2 transmit signal to R simultaneously
2nd time slot: R transmits the imposed signal to S1 and S2
• S1 and S2 know their own transmitted signal
• Self-interference can be removed in the receiver
60
Examples – Power Allocation Problem
 MIMO two-way relay system



Transmit filter of Sk : Tk = Tk Pk
• Power matrix : p k = diag (Pk )
Transmit filter of R : F = b −1 F
Receive filter of Sk : U kH = bPk−1 Wk U kH
Amplify-and-forward relaying in half-duplex mode
MIMO system with data steams, d k ∈  Lk ×1 where E {d k d kH } = I Lk
N ×N
Rayleigh flat fading channels: H k ∈  N R × Nkand G k ∈  k R
Complex additive white Gaussian noise (AWGN): n k and n R ~ CN ( 0, σ 2 I )
No direct path between S1 and S2 because of obstacles
H
The estimated data in the source =
Sk: dˆ k U k ( G k FH k Tk d k + G k Fn R + n k )
Power constraints
Source power constraint: p1 1 + p 2 1 ≤ PS
61
Examples – Power Allocation Problem
 Sum rate maximization in MIMO two-way relay system
1
Sum rate using relation between SINR and MMSE : SINRi + 1 =
MMSEi
 L1 + L2

1 L1 + L2
1 L1 + L2
1
log (1 + SINRi ) =
R=
− ∑ log ( MMSEi ) =
− log  ∏ MMSEi 
∑
2 i 1=
2 i1
2
=
 i =1

Optimization problem : sum-rate maximization under the power constraints
Sum-rate maximization problem is reformulated as a problem that minimizes
the product of the minimum mean square errors
L1 + L2
min
p1 ,p 2
∏ MMSE
i =1
i
s.t. 0 ≤ p1 1 + p 2 1 ≤ PS , pi > 0, ∀i
MSE of the i-th estimated data
(
)(
)
H
where [ Λ k ]ii =
MSEk ,i =E  d k , i − dˆk , i d k , i − dˆk , i 


=pk−1, i ( Λ k + Wk2 Ψ k ) p=
+ Wk2 N k 1Lk 
k
[ ψ k ]ij
i
{
1 − 2 Re wk , i u kH, i G k FH k t k , i
}
+ wk2, i u kH, i G k FH k t k , i t kH, i ( G k FH k ) u k , i
H
u kH, i G k FH k t k , j t kH, j ( G k FH k ) u k , i (1 − δ ij )
H
=
[ N k ]ii σ 2u kH, i ( G k F )( G k F ) u k , i + σ 2b
H
62
Examples – Power Allocation Problem
 Power allocation in MIMO two-way relay system
Power allocation problem
L1 + L2
min
p
∏ p ( Λ + W Ψ ) p + W N1
i =1
−1
i
2
s.t. 0 ≤ p 1 ≤ PS , pi > 0, ∀i
p = p1T , pT2 
2
i
T
Λ = blkdiag [ Λ 2 , Λ1 ]
ψ = blkdiag [ ψ 2 , ψ1 ]
W = blkdiag [ W2 , W1 ] N = blkdiag [ N 2 , N1 ]
Objective function and power constraints are the posynomials.
This optimization problem is a geometric program.
The GP (geometric programming) obtains the global optimum with
polynomial-time complexity through logarithmic transformations of the
non-convex problems to convex problems.
It can be solved by cvx tools.
63
Examples – Power Allocation Problem
 Power allocation in MIMO two-way relay system
CVX Code:
L1 + L2
min
p
∏ p ( Λ + W Ψ ) p + W N1
i =1
−1
i
2
2
i
64
Examples – Power Allocation Problem
 Power allocation in MIMO two-way relay system
Simulation results
The number of antennas at nodes : 4
Power allocation in two-way relay system
3
Equal power allocation
Power allocation with GP
2.5
Average sumrate
2
1.5
1
0.5
0
0
5
10
15
SNR [dB]
20
25
30
65
Examples – Support Vector Machine
 Support vector machine (SVM) in machine learning
Consider a classification problem
Given a set of data points (called feature vectors)
Each point is labelled +1 / -1.
Suppose we are given
a new point
• Where should this point
belong to?
4.5
class 1
class 2
4
3.5
3
2.5
2
4
4.5
5
5.5
6
6.5
7
7.5
8
66
Examples – Support Vector Machine
 Support vector machine (SVM) in machine learning
We would like to find a separating hyperplane for points x given by
wT x = b
For a new point x’, we classify by the sign
w T x '− b < 0
or w T x '− b > 0
Then, what is the best hyperplane ?
67
Examples – Support Vector Machine
 Support vector machine (SVM) in machine learning
Find hyperplane that gives the maximum margin between data sets
yi (w T xi − b)
maximize[min
]
i
w,b
|| w ||
T
T
yi = 1 if w xi − b > 0 // yi = −1 if w xi − b < 0
Rewrite the above problem:
maximize t
w ,b, t
maximize
w ,b, z
z
|| w ||
T
yi (w T xi − b)
subject to
≥ t z = t || w || subject to yi (w xi − b) ≥ t
|| w ||
|| w ||
minimize || w ||
w,b
w
z
b
b' =
z
w'=
subject to yi (wT xi − b) ≥ 1
This can be solved by CVX !
68
Examples – Support Vector Machine
 Support vector machine (SVM) in machine learning
Linear SVM code
% cvx
cvx_begin quiet
variables w(N_dim) b
minimize( norm(w) )
subject to
yi.*(w'*xi-b) >= 1;
cvx_end
4.5
SVM result class 1
SVM result class 2
source class 1
source class 2
4
3.5
3
2.5
2
4
4.5
5
5.5
6
6.5
7
7.5
8
Computed hyperplane
wT x = b
69
Examples – Support Vector Machine
 Support vector machine (SVM) in machine learning
Classification problem for Non-separable classes
Exact separation may not be possible.
We want to minimize the number of misclassified points.
• Add slack variables
minimize || w ||2 + C ∑ si
w ,b, s
4.5
class 1
class 2
4
i
subject to yi (w T xi − b) ≥ 1 − si
si ≥ 0
3.5
3
The slack variables quantifies
2.5
the violated classification.
C is regularization constant
(tunable parameter).
2
4
It can be solved by CVX.
4.5
5
5.5
6
6.5
7
7.5
8
Misclassified points
70
Examples – Support Vector Machine
 Support vector machine (SVM) in machine learning
SVM code
4.5
% cvx
cvx_begin quiet
variables w(N_dim) b si(N_tr)
minimize( w'*w + C*sum(si) ) 4
subject to
yi.*(w'*xi-b) +si' >= 1;
si >= 0;
3.5
cvx_end
SVM result class 1
SVM result class 2
source class 1
source class 2
3
2.5
2
4
4.5
5
5.5
6
6.5
7
7.5
8
Computed hyperplane
wT x = b
71