1. science
2. mathematics
3. statistics

Math 5652: Introduction to stochastic processes
Midterm 2, April 2, 2015 (003: afternoon section)
Solutions
1. (15 points) Define the following terms:
(a) Transition rate of a Markov chain
(b) Stationary distribution of a continuous-time Markov chain
(c) Alternating renewal process
Solution:
(a) For a time-homogeneous continuous-time Markov chain Xt , the transition rates are given
by
P(Xh = j|X0 = i)
ph (i, j)
= lim
.
q(i, j) = lim
h→0
h→0
h
h
(b) For a continuous-time Markov chain with transition rate matrix Q and transition probabilities
pt , a stationary distribution π is a probability distribution on the states (meaning
P
π(i)
= 1) such that πpt = π for all t, or equivalently πQ = 0.
i
(c) Given two sequences of iid positive random variables τ1 , τ2 , . . . and σ1 , σ2 , . . . (all independent, all the τ ’s have a common distrbution, all the σ’s have a common distribution),
an alternating renewal process counts the number of events that occur by time t if τi
is followed by σi . Formally, let the time between adjacent events be given by ρ1 = τ1 ,
ρ2 = σ1 , and generally ρ2k−1 = τk , ρ2k = σk , then the renewal process is given by
N (t) = max(n : ρ1 + . . . + ρn < t).
2. (15 points) Zillow is a website publishing advertisements for houses for sale. New houses
in the Twin Cities area appear on Zillow at a rate of 100 per day. The population of
the Twin Cities is approximately 1,000,000, and the average household size in the US is
approximately 2.5 people. Use this data to estimate the average length of time that a
household stays in a single house. State carefully any assumptions about the world
that you need to make – your answer will be graded based on those!
Solution: The computation itself is very simple. First, we find the number of houses in
the Twin Cities:
1household
106 people ·
= 4 × 105 houses.
2.5people
If 100 of them are getting sold every day, that means that the length of time that passes
between sales should be
4 × 105 houses
= 4000days,
100houses/day
or just under 11 years.
Here, we used Little’s law, with “system” being “houses being lived in”. Newly sold houses
arrive at the Zillow rate of 100/day, and the average number of houses in the system is
400, 000 houses.
The interesting part of the question is what assumptions we needed to make. Little’s law
assumes that we have an average number in the system, and average arrival rate into the
system. To quote the average number in the system, we need to assume that the number of
houses in Minneapolis hasn’t been changing. (That’s not actually true, since new houses
are being built – and old houses demolished – all the time.) We also tacitly assumed
that each household lives in a house, which is again not true: for example, Elena and her
husband (and probably many of you) rent an apartment. To quote the arrival rate of “new
houses being lived in”, we assumed that 100 houses per day is a long-term average; that
all houses that appear on Zillow get sold in reasonable time (so that 100 listings per day
also means 100 sales per day); and that all houses that are being sold appear on Zillow.
We are also assuming that houses are being lived in all the time between sales, which isn’t
entirely true (owners can move out but not sell the house).
3. (20 points) The MCFAM secretary has two meetings scheduled in one day: with the
academic director at 11am, and with the commercial director at 1pm. The first meeting
will last for an exponential time with mean 1.5 hours, and then the second meeting will last
for an exponential time with mean 2 hours. A meeting can’t start earlier than scheduled,
but if the first meeting runs over, she will finish it before starting the second one.
I will be going home at 3:00pm, but would like to meet with the secretary for 30 minutes.
What is the probability that I will find her free before her second meeting at 12:30pm?
What about at 2:30pm? (For 2:30pm, it is fine to leave an expression with integrals in it;
you do not need to evaluate the integrals.)
Let’s assume I don’t meet with the secretary, and she goes home when her meeting with
the commercial director ends. What is the expected time when the secretary goes home?
Solution: The probability that the secretary is free at 12:30 is the probability that her
first meeting took less than 1.5 hours, i.e. 1 − e−1.5/1.5 = 1 − e−1 ≈ 0.632.
To find the probability that she is free at 1:30, we subdivide into two cases. If her first
meeting finishes by 1pm (this happens with probability 1 − e−2/1.5 ), then conditionally on
that the probability of the second one being done by 1:30 is 1 − e−1.5/2 . If the first meeting
isn’t over by 1pm (this happens with probability e−2/1.5 ), then after 1pm I am waiting for
two exponential times: one with mean 1.5 hours, and the other with mean 2 hours. The
probability that their sum is less than 1.5 hours is
Z 1.5
1 −t/1.5
e
·
(1 − e−(1.5−t)/2 )
dt.
|
{z
}
1.5
0
| {z }
probability that 2nd one lasts at most 1.5 − t
density of time of first meeting
1
Thus, the probability that the secretary is free at 1:30 is
Z 1.5
1 −t/1.5
−2/1.5
−1.5/2
−2/1.5
(1 − e
)(1 − e
)+e
e
· (1 − e−(1.5−t)/2 )dt ≈ 0.445.
1.5
0
In particular, I have a better chance of catching the secretary at 11:30 than at 2:30.
The expected time for her to go home again requires splitting it up into the two cases:
does the first meeting end on time or no. If it does (with probability 1 − e−2/1.5 ), then
she goes home at expected time 1pm + E[E(1/2)] = 3pm. If it does not (with probability
e−2/1.5 ), then she goes home at expected time 1pm + E[E(1/1.5) + E(1/2)] = 4 : 30pm.
Consequently, the expected time she goes home is
(1 − e−2/1.5 ) · 3pm + e−2/1.5 · 4 : 30pm ≈ 3 : 24pm.
In converting time from decimals to minutes and back, remember there are 60 minutes in
an hour!
4. (20 points) On my way home, I pass by the Rainbow grocery store. It has a self-service
checkout lane and also a lane manned by a cashier. (Actually, it has more lanes, but that
would make the problem more complicated.) Customers arrive to the store and to the
check-out as a Poisson process with rate 5 per hour. It takes a customer exponential time
with mean 15 minutes to go through the self-service check-out, and exponential time with
mean 10 minutes to go through the lane manned by the cashier. If the cashier is free, the
arriving customer will always go to the cashier (even if the self-service lane is also free).
Find the stationary distribution for this system. What is the proportion of time when the
self-service lane is being used? (Your answer for the self-service lane should be numeric;
simplify any infinite sums in it.)
Solution: There was a mistake in the original solution – thanks to Job Boerma
for noticing it!
This is a variant of an M/M/2 queue, where the two servers have different service rates.
We will model it as a Markov chain. The state is the number of customers in the system,
but when there’s only one customer, we need to keep track of the lane he’s in. So we have
states 1 for cashier, and 1A for self-service: you go from 0 to 1 when a customer arrives,
but you might go from 2 to 1A when a customer leaves from the cashier. The transition
rates are

1
1

 10min + 15min = 10/hr, n ≥ 2


1


2 → 1A
 10min = 6/hr,
1
q(n, n + 1) = λ = 5/hr, q(n, n − 1) = 15min = 4/hr,
2→1


1

= 6/hr,
1→0

10min


 1 = 4/hr,
1A → 0.
15min
2
Here’s the diagram:
5
5
0
1
6
4
5
3
10
6
4
5
2
10
5
1A
Unlike the original solution that was up here, you can’t use detailed balance, because
there is a positive rate for going from 1A (self-service) to 0, but rate 0 for going back.
Instead, we use full balance:
π(0) · 5 = π(1) · 6 + π(1A) · 4
π(1) · 11 = π(0) · 5 + π(2) · 4
π(1A) · 9 = π(2) · 6
π(2) · 15 = π(1) · 5 + π(1A) · 5 + π(3) · 10
π(3) · 15 = π(2) · 5 + π(4) · 10
The other equations will look like the one for π(3), namely
π(n) · 15 = π(n − 1) · 5 + π(n + 1) · 10.
Notice that the first three equations have four variables among them: that means we can
solve for π(1), π(1A), and π(2) in terms of π(0), giving
5
π(1) = π(0),
8
π(1A) =
5
π(0),
16
π(2) =
5
π(0).
32
From the next equation,
π(3) =
15
1
(15π(2) − 5π(1) − 5π(1A)) = π(0).
10
64
For the one after that,
π(4) =
1
15
1
(15π(3) − 5π(2)) =
π(0) = π(3).
10
128
2
We see that for later states, π(n + 1) = 12 π(n) is going to work. Intuitively this is because
this ratio satisfies the detailed balance for all the subsequent edges; and more to the point,
if π(n + 1) = 21 π(n) = 14 π(n − 1) then the full balance equation is satisfied:
5π(n − 1) + 10π(n + 1) = 10π(n) + 5π(n) = 15π(n)
as required.
3
Finally we need to solve for π(0):
5
5
15
15
5
+
+
+
+ ... = 1
π(0) 1 + +
8 16 32 64 128
The sum is equal to
1+
5
5
5
15
5
5
5
15
41
+
+
+
· (1 + 1/2 + 1/4 + . . . ) = 1 + +
+
+
= ,
8 16 32 64
8 16 32 32
16
from which π(0) =
π(0) =
16
,
41
16
,
41
and the rest of the states follow:
π(1) =
10
,
41
π(1A) =
5
,
41
π(2) =
5
,
82
π(3) =
15
164
and from then onwards, π(n + 1) = 12 π(n).
The self-service lane is being used unless we’re in state 0 or 1, so
1 − π(0) − π(1) = 1 −
16 10
15
−
= .
41 41
41
5. (20 points) I am currently working on three research projects; let’s call them (by names
of my collaborators) Carlos, Frank, and George. If I spend too long on one project, I run
out of imagination, and switch to a different one. I spend an exponential time with mean
1 day on project Carlos, and then switch to one of the other two with equal probability.
When working on project Frank, I spend an exponential time with mean 3 days on it, and
switch to one of the other two with equal probability. Finally, when working on project
George, I spend an exponential time with mean 4 days on it, and then switch to one of
the other two with equal probability.
What is the long-term fraction of time when I work on project Carlos?
Solution: This is a 3-state Markov chain, with transition matrix
C
F
G

C −1 1/2
1/2
F  1/6 −1/3 1/6 
G 1/8 1/8 −1/4

The stationary distribution satisfies
1
1
π(C) · 1 = π(F ) + π(G)
6
8
1
1
π(F ) · 1/3 = π(C) + π(G)
2
8
1
1
π(G) · 1/4 = π(C) + π(F )
2
6
π(C) + π(F ) + π(G) = 1
4
(One of the first three equations is redundant.)
Solving this system of equations, I find
1 3 1
π = ( , , ),
8 8 2
so in the long run, about 1/8 of my time is spent working on project Carlos.
5