APPM 2360
Lab #2: Facial Recognition
Instructions
Labs may be done in groups of 3 or less. You may use any program; but the TAs
will only answer coding questions in MATLAB. One report must be turned in
for each group and must be in PDF format. Labs must include each student’s:
• Name
• Student number
• Section number
• Recitation number
This lab is due on Friday, March 20, 2015 at 5pm. Each lab must be
turned in through D2L on the course page. When you submit the lab please include each group members information (name, student number, section number
and recitation number) in the comments. This allows us to search for a students
report. Once the labs are graded you need to specifically check that your grade
was entered. If you are missing a grade please bring the issue up with your TA
within a week of grading.
1
The report must be typed (including equations). Be sure to label all images
and graphs so that, independent of the text, it is clear what is in the graph.
Simply answering the lab questions will not earn you a good grade. Take time
to write your report as up to 20% of your grade may be based on organization,
structure, style, grammar, and spelling. Project office hours will be held in
ECCR 143 from Monday, March 16, 2015 to Friday, March 20, 2015. (Except
that on Tuesday March 17, office hours will be in the Stadium as usual.)
1
Objective
In this lab we use the singular value decomposition for matrices. We use it to
build a facial recognition system.
2
Matrix and Image Manipulation in Matlab
It’s easy to read grayscale images into Matlab, using the function imread. Matlab stores the image as a matrix. Each entry is an integer between 0 and 255.
An entry of 0 corresponds to one black pixel; an entry of 255 corresponds to
one white pixel.
Download the images from the folder faces included with the lab. Make sure
the images are in your current Matlab folder. Then try these Matlab commands:
A = imread('face1.pgm');
figure;
imshow(A);
Next, we want to try manipulating the matrix and then viewing the image
again. First, we need to change the datatype of the Matrix to double. Then we
can do matrix operations. Then we change the datatype back to uint8 before
we use imshow. For example:
A = imread('face1.pgm');
A = double(A);
newA = -A+255;
newA = uint8(newA);
figure;
imshow(newA);
Problem 1. Try the code above, and include the resulting figure in your report.
Describe what has happened to the image.
In case you are unfamiliar with matrices in Matlab, here are some things to
know:
• If two matrices A and B are the same shape, you can add them in Matlab
by typing A + B.
2
• If A and B are the right shape to be multiplied, Matlab will multiply them
with A ∗ B.
• You can take the transpose of a matrix with an apostrophe: A0 is the
transpose of A.
• You can get the diagonal elements out of a matrix A using diag(A).
Singular Value Decomposition
The Singular Value Decomposition breaks a matrix A down into three
separate matrices A = U SV T where V T denotes the transpose of V . The
matrices U and V are both orthogonal, which means that U T U = I and V T V =
I, where I is the identity matrix. The matrix S is diagonal; and the diagonal
elements of S are called the singular values of A.
Intuitively, the matrices U and V describe important features of the matrix,
and the matrix S gives us a numerical measure of how important those features
are. Don’t worry about the technical details for now - we’ll get a feel for what
this decomposition does as we continue to work with it.
Below is a brief MATLAB script that takes the SVD of a random 3×3 matrix.
After you run it, take a look at what U , S, and V look like!
%% Test Matlab's SVD function
% Build a random 3x3 matrix
A = rand(3);
% Get singular value decomposition
[U,S,V] = svd(A);
% Multiply to get new matrix
B = U*S*V';
%Check that we got the same thing we started with
A
B
Low-Dimensional Approximations
What does it mean to create a low-dimensional approximation to a set of data?
Take a look at Figure 1. The data points shown on the left lie in two dimensions,
but we can easily see that they are “almost” one-dimensional, i.e. they are all
pretty close to some line. If we project each point onto this line (shown in blue),
we end up with a one-dimensional approximation of the data. Obviously, we
have lost some information, but the hope is that most of the important aspects
of the data are preserved in this process.
The SVD allows us to do a low-dimensional approximation of a matrix. Suppose
we take the SVD of a matrix. The diagonal of S will be the singular values,
which MATLAB automatically ranks in descending order, so that we might have
something like
3
Figure 1: A low-dimension approximation to a set of points.
diag(S) = {60.4, 12, 1.07, 0.9, 0.05, 0.001, 0.00001}
In the last section, we said that these singular values can be thought of as
weighting how important the corresponding features are. So, it seems like the
last three features are relatively unimportant, doesn’t it? Supposed we replaced
S with Se where
e = {60.4, 12, 1.07, 0.9, 0, 0, 0}
diag(S)
We have set all but the first 4 singular values to zero.
Then we set
e = U ∗ Se ∗ V T
A
e is 4, down from 7 originally.1 However A
e is a good
The “dimension” of A
approximation of our A, since the features we are ignoring are relatively unimportant.
In the next section, we’ll explore removing the less important singular values
and observe the effects. Remember, the most important singular values are
always the first ones.
Making Low-Dimensional Approximations to Images
Run this code in Matlab:
%%
A = imread('face1.pgm');
A = double(A);
[U,S,V] = svd(A);
1 Technically,
when we say “dimension” here we really mean rank.
4
% Make a copy of S
Sreduced = S;
% Turn all but the first 10 singular values to 0
Sreduced(10:end,10:end) = 0;
% remultiply to get lower-dimensional version of A
Areduced = U*Sreduced*V';
figure;
imshow(uint8(A));
figure;
imshow(uint8(Areduced));
(a) All the singular values
(b) Only 10 singular values
Pretty cool! We reduced the dimension all the way down to 10, and we can
still see the important features of the face.
Problem 2. Follow the above Matlab procedure to project images to lower dimensions. Work with face1.pgm, and try a variety of values of r, where r is
the number of singular value you keep. Try keeping most of the singular values,
or half, or only a few. Include these figures in your report (try at least 3, and
make sure they look different).
Problem 3. Follow the above Matlab procedure for 3 of the other faces. How
low can you make r while still being able to recognize the face? Include some
figures to demonstrate.
Problem 4. Find all the singular values for face1.pgm, . Produce a plot of
these values. How many of the singular values are larger than 0.01?
Facial Recognition
Suppose the FBI is searching for a criminal on the run. The identity of the
criminal is unknown. However, the FBI suspects that the criminal will try to
obtain a new driver’s license under a fake name.
5
The criminal will enter a DMV (Department of Motor Vehicles, in charge of
licensing drivers) and take a photo for a new license. The FBI hopes to catch
the criminal by comparing the new photo to a database containing all the other
driver’s license photos in the US. When they see that the same face is already
in the database under a different name, they’ll be able to identify the criminal.
The process of searching through the database for a match needs to go very
quickly, because the database is huge. The FBI wants to find the match before
the criminal leaves the DMV, so they can rush in and make an arrest.
The FBI hires you to design an algorithm that can identify matching faces
very quickly. When a new photo is taken at the DMV, you must compare it to
all the existing photos and look for a matching face. If you compare the photos
pixel by pixel, you won’t be nearly fast enough.
It turns out that you can use the SVD to very quickly compare photos. For
each photo, simply use the vector of the first r singular values, where r is some
suitably chosen integer. This comparatively short vector of numbers captures
the most important information about each face. This is called the “signature
vector” for each face.
Your strategy is to first make a signature vector for each face in the database.
When a new photo is taken at a DMV somewhere in the country, you’ll make
a new signature vector for it. You’ll compare the new signature vector to the
old ones in the database. If the new signature vector is close to any of the old
ones, you’ll alert the FBI. This algorithm is much faster than comparing all the
pixels in the photograph.
This method works best if the photographs in your database are all taken
under similar conditions (similar lighting, similar zoom, face centered). This is
true for driver’s license photos, so we’re in luck. If photo conditions were more
variable, we would have to adjust for those variations first. .
Problem 5. Set r = 15. Make a signature vector for each of the 6 images
face1.pgm through face6.pgm Include the signature vectors for images 5 and
6 in your lab report. Also make a signature vector for the image newFace.pgm.
In the next problem you will use the signature vectors to test whether newFace
matches any of the other faces.
We want to know whether the signature vector for newFace is close to any
of the other signature vectors. What does it mean for vectors to be close? You
already know how
p to calculate the distance between points in 3 dimensions: that
is, distance = (x1 − x2 )2 + (y1 − y2 )2 + (z1 − z2 )2 . There’s a similar distance
function for our signature vectors, except in 15 dimensions instead of 3.
The Matlab function norm calculates the length of a vector. To find the
distance between two vectors, we calculate the length of their difference. In
Matlab that will look like norm(vector1-vector2).
Problem 6. Find the distance between the signature vector for newFace and
each of the other signature vectors. In your report, include a plot or graph that
shows each of these 6 distances. Which image is newFace closest to?
6
Problem 7. Given this signature vector:
12830
11258
1100
890
640
580
415
370
333
300
290
250
240
Which of the 6 faces is this closest to?
Problem 8. Suppose your job is to build a facial recognition system using SVD,
for a database with 100,000 faces. Describe how you would choose the best value
of r. What would happen if it was too small? What would happen if it was too
large?
7
220
215