1. pets
2. birds

Math 5652: Introduction to stochastic processes
Midterm 2, April 2, 2015 (001: morning section)
Solution
1. (15 points) Define the following terms:
(a) Exponential random variable
(b) Renewal process
(c) Stationary distribution of a continuous-time Markov chain
Solution:
(a) An exponential random variable is a continuous random variable with density
fX (x) = λe−λx ,
x ≥ 0;
fX (x) = 0,
x<0
for some parameter λ > 0.
(b) A renewal process N (t) counts the number of events that occur by time t. More precisely,
if τ1 , τ2 , . . . are iid positive random variables, then N (t) = max{n : τ1 + . . . + τn ≤ t}.
(Sometimes we allow τi to take the value 0 with positive probability, but that’s rare.)
(c) For a continuous-time Markov chain with transition rate matrix Q and transition probabilities
pt , a stationary distribution π is a probability distribution on the states (meaning
P
i π(i) = 1) such that πpt = π for all t, or equivalently πQ = 0.
2. (15 points) During spring thaws, at Barn Bluff in Redwing, Minnesota, rockfalls happen
as a Poisson process with rate 2 per week.
Climbers come to Redwing on days when it is sunny. The weather pattern for Redwing
alternates between sunny periods that on average last for 3 days, and rainy spells that on
average last for 2 days; lengths of different spells are independent. On sunny days, it is
light for 12 hours of the day (i.e. half the time); climbers only come when it is light.
If any climbers are present at Redwing during a rockfall, the probability that someone is
injured is 1/6. Find the average number of people injured during the 3 months (13 weeks)
of spring.
Solution: First, let’s figure out what proportion of the time climbers are at Redwing.
The weather process is an alternating renewal process, so the proportion of days on which
it is sunny is 3/(3 + 2) = 3/5. Of those days, climbers are there for only half of the time,
so the proportion of time when there are climbers at Redwing is 0.3.
Consequently (by the PASTA property), 30% of rockfalls happen when climbers are
present, and 1/6 of those injure someone, so the rate of injuring climbers is 0.3 · 2/week ·
1/6 = 0.1/week. In 13 weeks, therefore, we expect
13 · 0.1 = 1.3injuries,
so we expect 1 or 2 people to get injured by falling rocks every year.
3. (20 points) A home inspector has two appointments in the same neighborhood scheduled
in one day: the first is at 9am, and the second is at 1pm. The appointment cannot start
earlier than it is scheduled. Each inspection lasts for a time that is exponential with mean
3 hours; the inspector finishes each inspection before moving on to the next one. (Ignore
travel time between appointments.)
(a) What is the expected time when the inspector finishes the two inspections?
(b) What is the probability that the inspector finishes both inspections by 4pm? Write
down an expression (which may involve integrals); you do simplify your answer or give
a numeric value.
Solution:
(a) With probability 1 − e−4/3 , the first appointment ends before 1pm, and the inspector
starts the 2nd appointment on time: conditional on the first appointment ending on
time, the expected end of the second one is 4pm.
With the remaining probability e−4/3 , the first appointment is still going on at 1pm, and
then it will continue for another exponential time with mean 3 hours, after which the
second appointment will start. In this case, the expected end of the second appointment
is 7pm.
Therefore, the expected ending time is
(1 − e−4/3 ) · 4pm + e−4/3 · 7pm ≈ 4 : 47pm.
(b) We again partition the possibilities into whether the first appointment ended by 1pm
or no.
If it did (with probability 1−e−4/3 ), the probability of ending by 4pm 1−e−3/3 = 1−e−1 .
If it didn’t (with probability e−4/3 ), the probability of ending by 4pm is the probability
that the sum of two exponentials is smaller than 3. Conditioning on how long after 1pm
the first inspection actually takes, we get
Z 3
1 −t/3
·
(1 − e−(3−t)/3 )
e
dt
|
{z
}
0
|3 {z }
probability that 2nd one lasts at most 3 − t
density of time of first inspection
The final answer is then
(1−e
−4/3
−1
)(1−e )+e
−4/3
Z
0
3
1 −t/3
e
(1−e−(3−t)/3 )dt = (1−e−4/3 )(1−e−1 )+e−4/3 (1−2e−1 ) ≈ 0.535.
3
1
4. (20 points) The math department has one secretary who processes travel paperwork (professors are sometimes reimbursed by the university for travel to conferences). She is able to
process 5 travel applications per day. Around the time of a major conference (AMS/MAA
joint meetings, for example), professors come with reimbursement requests at a rate of 3
a day. Any reimbursement requests that the secretary hasn’t processed yet are sitting on
her desk (including the one she is currently working on).
Assuming professors come as a Poisson process, and processing an application takes an
exponentially distributed amount of time, find the stationary distribution for the number
of reimbursement requests that are sitting on the professor’s desk. What is the probability
that the number is bigger than 5? (Simplify any infinite sums in your answer.)
Solution: We model this as an M/M/1 queue, i.e. an infinite state-space Markov chain.
The state of the chain is the number of applications on the secretary’s desk. The transition
rates are
q(n, n + 1) = 3,
q(n, n − 1) = 5 if n ≥ 1.
Detailed balance equations read
3
π(n) · 3 = π(n + 1) · 5 =⇒ π(n + 1) = π(n) = . . . =
5
n+1
3
π(0).
5
That is, the stationary distribution is a geometric progression. The value of π(0) can
found from the fact that the stationary distribution should sum to 1:
1
3
3
3 2
2
= .
π(0) 1 + + ( ) + . . . = 1 =⇒ π(0)
3 = 1 =⇒ π(0) = 1 −
5
5
5
5
1− 5
(Alternatively, we know that the secretary must be busy a fraction 3/5 of the time, hence
π(0) = 1 − 3/5.)
We then conclude π(n) =
2
5
· ( 35 )n .
The probability that there are more than 5 applications on the secretary’s desk is
X2
n≥6
3
2 3
3
3
· ( )n = · ( )6 · (1 + + ( )2 + . . . )
5 5
5 5
5
5
=
2 3 6
1
·( ) ·
5 5
1−
3
5
3
= ( )6 ≈ 0.047.
5
We might, of course, recognize this as the tail probability of a geometric distribution,
and then skip the calculation (in the same way as we know the tail distribution of the
exponential distribution).
5. (20 points) I have two different birdfeeders. Food in the “house” feeder lasts for a time
that’s exponential with mean 3 days, while food in the “suet” feeder lasts for a time
2
that’s exponential with mean 6 days. I check the status of the feeders after a time that’s
exponential with mean 9 days; if at that point any of them need filling, I refill them.
Model this system as a four-state Markov chain. You can write down the transition matrix,
or draw a diagram that specifies all transition rates. Measure all rates as number per day.
Sparrows come to my balcony as long as at least one of the birdfeeders has food in it,
while chickadees will only come if both feeders still have food in them. Set up, but do
not solve a system of equations that lets you find the proportion of time when sparrows
and chickadees come to my window. Be sure to indicate how the answer is related to the
solution of your system of equations!
Solution: The states will be the set of birdfeeders that have food in them: so None,
House, Suet, or Both. The transition rate matrix is
N
H
S
B

N −1/9
0
0
1/9

H
0
1/9
 1/3 −(1/3 + 1/9)

Q=

S  1/6
0
−(1/6 + 1/9)
1/9
B
0
1/6
1/3
−(1/3 + 1/6)

In words, if not both feeders are full, then I transition to state B at rate 1/9 (mean time
79days), and each feeder empties out at rate 1/3 or 1/6 depending on whether it’s House
or Suet. If both feeders are full, I only transition to “one feeder empty” states: I can’t
empty out both feeders simultaneously. The diagonal entries make each row add up to 0.
To figure out the proportion of time I see both types of birds, I will need the stationary
distribution of this Markov chain. The stationary distribution satisfies the following system
of equations:
π(N ) · 1/9 = π(H) · 1/3 + π(S) · 1/6
π(H) · (1/3 + 1/9) = π(B) · 1/6
π(S) · (1/6 + 1/9) = π(B) · 1/3
π(B) · (1/3 + 1/6) = π(N ) · 1/9 + π(H) · 1/9
π(N ) + π(H) + π(S) + π(B) = 1
Of the first four equations, one is redundant.
Solving the equations gives
π=
2 3 117 12
, ,
,
11 44 220 55
,
but you don’t have to do that.
In terms of the solutions to the equations, the proportion of time sparrows come is 1 −
π(N ) ≈ 0.82, and the proportion of time chickadees come is π(B) ≈ 0.22.
3