1. science
2. mathematics
3. statistics

Math 5652: Introduction to Stochastic Processes
Practice midterm 2: solutions
The midterm will be closed-book and closed-notes. It covers the following sections from the
textbook:
2.1, 2.2, 2.4; 3.1, 3.2; 4.1, 4.2, 4.3.
I may ask you to define certain terms – use the end-of-chapter summaries in Durrett to
look them up. For example, here are some things I might ask: Poisson process, exponential
random variable, renewal process, Little’s law (you’ll need to clearly state what the three
components of it are!), Markov property in continuous time, continuous-time Markov chain.
This is not exhaustive; use the end-of-chapter summaries in Durrett to see all the terms that
have been defined in the chapter.
Calculators are allowed but should not be necessary. You do not need to simplify numeric
answers.
(1) Let N (t) be a Poisson process with rate 2, and let T12 be the time of the 12th arrival.
Find E[T12 |N (2) = 5].
Solution: Let τi be the iid E(2) times between the events of the Poisson process,
then
E[T12 |N (2) = 5] = E[τ1 + . . . + τ12 |τ1 + . . . + τ5 ≤ 2, τ2 + . . . + τ6 ≥ 2]
1
= 5.5
2
In words, I need to wait for 7 more arrivals after time 2, so the time is 2 + the
expected time for 7 arrivals.
= 2 + E[τ6 + . . . + τ12 ] = 2 + 7 ·
(2) Let N (t) be a Poisson process with rate 2. Let TK be the first time that the gap
between events is at least 3, i.e.
K = min{n ≥ 1 : Tn − Tn−1 ≥ 3}.
Find the distribution of K. What is E[K]?
Slightly harder: what is E[TK ]?
Solution: For each interval between events of a Poisson process, the probability of
its being bigger than 3 is e−2·3 . Consequently,
P(K = k) = (1 − e−6 )k−1 e−6 ,
since the first k − 1 intervals need to be shorter than 3, and the last one needs to be
longer. This is a geometric distribution.
The mean of the geometric distribution is the inverse of the probability, in this case
E[K] = 1/(e−6 ) = e6 . If you’ve never done the computation, here’s how it goes.
Consider a geometric distribution with parameter p, then to find its mean we need
to compute
1 · (1 − p) + 2 · p(1 − p) + 3 · p2 (1 − p) + . . . = (1 − p) 1 + 2p + 3p2 + 4p3 + . . .
1
The infinite sum isn’t a geometric series, but it’s the derivative of one: if we let
f (p) = 1 + 2p + 3p2 + . . . , then
Z
1
−1
f (p)dp = p + p2 + p3 + . . . =
1−p
(the −1 is because this geometric series doesn’t start with 1), and therefore
d
1
1
f (p) =
−1 =
.
dp 1 − p
(1 − p)2
Remember that we wanted (1 − p)f (p) = 1/(1 − p); in this case 1 − p = e−6 .
(More intuitively, if the probability of success on a trial is e−6 then the number of
trials you need to succeed is the inverse of that, or e6 .)
Going back to E[TK ], we can think of it as the sum of a random number of random
times: we are adding up K − 1 times that are iid exponential conditioned on being
< 3, and then one time that is exponential conditioned on being ≥ 3. As we saw
on the homework, the mean of the sum of K − 1 iid random variables is E[K − 1]
times the mean of one of them, which means that we just need to find the mean of
an exponential conditioned on being < 3, and the mean of it conditioned on being
≥ 3.
Now,
E[E(2)|E(2) > 3] = E[3 + E(2)] = 3.5 :
if we know an exponential random variable is at least 3, then the excess amount is
another exponential random variable with the same parameter. We also know that
E[E(2)] = 21 . Consequently,
E[E(2)|E(2) < 3] P(E(2) < 3) + E[E(2)|E(2) > 3] P(E(2) > 3) = E[E(2)]
|
{z
} |
{z
}|
{z
} | {z }
1−e−6
from which E[E(2)|E(2) < 3] =
e−6
3.5
e6 −7
1
2 e6 −1
1/2
≈ 0.49254. Thus,
E[TK ] = E[K − 1] · E[E(2)|E(2) < 3] + 1 · E[E(2)|E(2) > 3]
= (e6 − 1)
1
1 e6 − 7
+ 3.5 = e6 .
6
2e −1
2
Another way to get at this answer is to see that the process that counts only those
events where the time immediately before the event was longer than 3 is a Poisson
process with rate 2e−6 , because it is a thinning of the original Poisson process. Therefore, the time until the first event in this thinned Poisson process is exponential with
parameter 2e−6 and mean 12 e6 .
(3) Power surges can be damaging to computers that don’t have a surge protector. Surges
are classified into three types: “small”, “medium”, and “large”. Small surges occur
at a rate of once every 3 hours (about 8 per day), but the probability that it will
damage a computer is 0.001. Medium surges happen, on average, once a day; the
probability that a medium surge will damage a computer is 0.01. Large surges occur
2
about once a month, but the probability that one damages a computer is 0.1. Assume
that the three types of surges occur as independent Poisson processes.
(a) What is the expected number of computer-damaging power surges in one month?
(b) What is the probability that the first power surge that damages a computer is
a small surge?
Solution: Notice that the computer-damaging surges within each of the “small”,
“medium”, and “large” types are themselves Poisson processes, because they are
thinned versions of the original. The superposition of the “small computer-damaging”,
“medium computer-damaging”, and “large computer-damaging” is therefore a Poisson process.
I will treat a month as being equal to 30 days.
(a) The rate of that Poisson process is
8/day · 0.001 + 1/day · 0.01 + 1/30days · 0.1 ≈ 0.0213/day.
Assuming a month is 30 days, that’s a rate of about 0.64 computer-damaging
surges per month. That is also the expected number of computer-damaging
power surges in a month.
(b) If we look at the times until the first small computer-damaging power surge,
first medium, and first large, this is asking about the probability that the first
of those is the “first small computer-damaging power surge”. The probability
that the minimum of three exponential random variables is the “small computerdamaging” one is
8/day · 0.001
= 0.375.
8/day · 0.001 + 1/day · 0.01 + 1/30days · 0.1
(4) Let T1 , T2 , ... be iid random variables with density
fT (t) = te−t .
Let
N (t) = max{n : T1 + T2 + . . . + Tn ≤ T }.
Find
N (t)
.
t→∞
t
lim
Solution: This set-up exactly identifies N (t) as a renewal process, with times between events having density fT . By the renewal theory,
N (t)
1
1
= R∞
.
=
t→∞
t
E[Ti ]
t · te−t dt
0
lim
Integration by parts gives the value of the integral as 2, so the answer is 1/2. (The
density is actually the density of the sum of two independent exponentials of rate 1,
hence the mean time between events is indeed 2.)
3
(5) An auto insurance company classifies its customers as “poor”, “satisfactory”, and
“excellent”. The transition rate matrix (per year) between these categories is given
by
P
S
E


P
x 0.4 0
S  0.1 y 0.3 
E 0 0.2 z
(a) Find x, y, and z.
(b) Determine the long-term fraction of excellent customers the insurance company
has.
(c) How long, on average, does it take for a “poor” customer to be promoted to
“satisfactory”?
(d) What is the probability that when a “satisfactory” customer changes status, he
is upgraded to “excellent” rather than downgraded to “poor”?
Solution:
(a) Rows of the Q matrix need to add up to 0, so x = −0.4, y = −0.4, and z = −0.2.
(b) The system of equations for stationarity reads
π(P ) · 0.4 = π(S) · 0.1
π(S) · 0.4 = π(P ) · 0.4 + π(E) · 0.2
π(E) · 0.2 = π(S) · 0.3
π(P ) + π(S) + π(E) = 1
Solving this system gives
1 4 6
, , ).
11 11 11
Thus, the long-term fraction of excellent customers is π(E) = 6/11 ≈ 0.545.
π=(
(c) Customer leaves the “poor” state after a time that’s exponential with parameter
−x = 0.4, i.e. mean of 2.5 years. Since customers leaving the “poor” state go to
the “satisfactory” state, the answer is 2.5 years.
(d) The probability in question is q(S, E)/(−q(S, S)) = 0.3/0.4 = 3/4.
4