AMS 206: Classical and Bayesian Inference
Exam 2 solutions
1. Assume that X1 , ..., Xn is a random sample from a normal distribution with unknown mean µ
and known variance σ 2 = 1. How large must the sample size n be in order for the confidence interval
for µ, with confidence coefficient 0.9, to have length less than 0.1? (It is given that Φ(1.645) = 0.95,
that is, the value of the standard normal distribution function at 1.645 is 0.95.)
√
√
Solution: The 90% confidence interval for µ is given by (¯
xn − (cσ/ n), x
¯n + (cσ/ n)), where
σ = 1 and c = 1.645, using the information from the problem statement. Therefore, the length
√
of the 90% confidence interval for µ is 3.29/ n, and for this to be less than 0.1, we must have
n > 1082.41, that is, the sample size must be greater than or equal to 1083.
2. Suppose that, conditional on (unknown) mean parameter θ > 0, X1 , ..., Xn form a random
sample from a Poisson distribution with probability function
f (x | θ) =
e−θ θx
,
x!
x = 0, 1, 2, ...
(a) What is the posterior distribution for θ under an exponential prior distribution with mean 1?
Solution: We have shown that the posterior distribution for θ under a gamma(α, β) prior is
given by a gamma distribution with shape parameter α + n¯
xn and rate parameter β + n.
The exponential prior is the special case of the gamma(α, β) prior with α = β = 1, and therefore,
the posterior distribution for θ is a gamma distribution with shape parameter 1 + n¯
xn and rate
parameter 1 + n.
(b) Consider a specific data set, with sample size n = 10 and sample mean 201.8, assumed to
arise as a random sample from a Poisson distribution with mean θ. Compare for this data set the
M.L.E. θb = x
¯n with the posterior expectation for θ, the latter obtained under the prior from part
(a). What is the reason (or reasons) for the difference between the values of the M.L.E. and the
posterior expectation?
Solution: The M.L.E. is x
¯n = 201.8, and the posterior expectation
1 + n¯
xn
n−1 + x
¯n
= −1
= 183.55.
1+n
n +1
One reason for the difference between the values of the M.L.E. and the posterior expectation is
that the exponential prior distribution is fairly incompatible with the data; note that the prior
expectation for the Poisson mean is α/β = 1, whereas the sample mean is 201.8. Another reason
is the small sample size. Note that, with the same exponential prior and for data sets with the
same sample mean, the posterior expectation will get closer to the M.L. estimate as the the sample
size increases. For instance, with n = 100, E(θ | x1 , ..., xn ) = 199.81, and with n = 1000, E(θ |
x1 , ..., xn ) = 201.6.
E(θ | x1 , ..., xn ) =
3. Suppose that X1 , ..., Xn form a random sample from the Rayleigh distribution, which is a
continuous distribution with probability density function
f (x | θ) = θx exp(−0.5θx2 ),
for x > 0
where θ > 0 is the (unknown) parameter of the distribution.
(a) Obtain the Fisher information In (θ) in the random sample (X1 , ..., Xn ).
Solution: The regularity conditions are satisfied for the Rayleigh distribution (the sample space
does not depend on θ), and therefore In (θ) = nI(θ), where I(θ) = E(−∂ 2 log f (x | θ)/∂θ2 ) =
E(1/θ2 ) = 1/θ2 . Hence, In (θ) = n/θ2 .
(b) Derive the M.L.E. of θ. Obtain the asymptotic confidence interval, with confidence coefficient
γ, for θ based on the large-sample normal approximation to the distribution of the maximum likelihood estimator of θ.
P
asympSolution: The M.L.E. was obtained as part of the first exam, θbn = 2n/( ni=1 x2i ). Thep
totic distribution of the M.E. estimator is normal with mean θ, and standard deviation 1/ In (θ).
b the standard deviation beUsing the result from part (a), and estimating θ by the M.L. estimate θ,
√
b n. Therefore, the asymptotic confidence interval with confidence coefficient γ is given by
comes θ/
√
b
b
b √n)), where c is the (1 + γ)/2 percentile of the standard normal distribution,
(θ − (c θ/ n), θb+ (c θ/
that is, Φ(c) = (1 + γ)/2.
4. Consider Bayesian inference for the same distribution with problem 3, that is, now conditional
on θ > 0, the X1 , ..., Xn form a random sample from the Rayleigh distribution with probability
density function f (x | θ) = θx exp(−0.5θx2 ), for x > 0. Consider a gamma distribution with parameters α and β as the prior distribution for θ, that is, ξ(θ) ∝ θα−1 exp(−βθ), for θ > 0.
(a) Show that the gamma distribution above provides the conjugate prior for θ, and obtain the
parameters of the posterior distribution given data x = (x1 , ..., xn ).
Solution: The posterior distribution can be written proportional to
Xn
Xn
ξ(θ | x) ∝ θn exp −0.5θ
x2i θα−1 exp(−βθ) ∝ θ(n+α)−1 exp −(β + 0.5
x2i )θ
i=1
i=1
which can beP
recognized as a gamma distribution with posterior hyperparameters α∗ = α + n and
∗
β = β + 0.5 ni=1 x2i .
(b) Obtain the posterior expectation for the median of the Rayleigh distribution.
Solution: As derived for problem 3 of exam 1, the median of the Rayleigh distribution is η(θ) =
(2 log(2))1/2 /θ1/2 . Therefore,
E(η(θ) | x) = (2 log(2))1/2
=
=
R∞
0
θ−1/2 ξ(θ | x) dθ
∗
(2 log(2))1/2 β ∗α
R ∞ (α∗ −0.5)−1
exp(−β ∗ θ) dθ
0 θ
Γ(α∗ )
1/2
(2 log(2)) Γ(n+α−0.5)
P
2 −1/2 Γ(n+α)
(β+0.5 n
i=1 xi )
where the integral in the second line is obtained through the normalizing constant of a gamma
distribution with parameters α∗ − 0.5 = α + n − 0.5 > 0 and β ∗ .
(c) Derive the expression for the posterior predictive density, f (xn+1 | x), corresponding to future
response Xn+1 with (unobserved) value xn+1 .
Solution: Using the definition of the posterior predictive density,
R
f (xn+1 | x) =
f (xn+1 | θ)ξ(θ | x) dθ
∗
R∞
β ∗α
α∗ −1 exp(−β ∗ θ) dθ
= 0 θxn+1 exp(−0.5θx2n+1 ) Γ(α
∗) θ
R ∞ (α∗ +1)−1
∗α∗
exp(−(β ∗ + 0.5x2n+1 )θ) dθ
= β Γ(αx∗n+1
0 θ
)
∗
=
=
Γ(α∗ +1)
β ∗α xn+1
Γ(α∗ ) (β ∗ +0.5x2n+1 )α∗ +1
P
2 n+α x
(n+α)(β+0.5 n
n+1
i=1 xi )
Pn
,
((β+0.5 i=1 x2i )+0.5x2n+1 )n+α+1
for xn+1 > 0
where the integral in the third line is evaluated using the fact that the integrand is proportional
to the density
a gamma distribution with parameters α∗ + 1 = α + n + 1 and β ∗ + 0.5x2n+1 =
Pn of
(β + 0.5 i=1 x2i ) + 0.5x2n+1 .