Problem Set #3: Estimation and finite sample inference Economics 835: Econometrics Fall 2012 1 Averages Let x1 , x2 , . . . , xn be a random sample of size n on the random variable x. Let µ = E(x) and σ 2 = var(x). Let a1 , a2 , . . . , an be a sequence of n numbers. a) Let: Wn = a1 x1 + a2 x2 + · · · + an xn Find a condition on the a’s that imply that Wn is an unbiased estimator of µ. b) Find var(Wn ) as a function of the a’s and of σ 2 . c) For any set of numbers a1 , a2 , . . . , an it is the case that: (a1 + a2 + · · · + an )2 ≤ a21 + a22 + · · · + a2n n ¯ n ), where X ¯ n is the usual Use this to show that if Wn is an unbiased estimator of µ, then var(Wn ) ≥ var(X sample average. 2 Inference on the median One advantage of the median over the mean is that yields exact small-sample inference even without assuming x is normally distributed. Let x be a random variable with an unknown continuous probability distribution, and suppose that we have a random sample of size n on that random variable. We would like to construct a 95% confidence interval for the median M a) We start by constructing a hypothesis test. Suppose we are testing the null that the median of x is M against the alternative that it is not. H0 : Pr(x ≤ M ) = 0.5 H1 : Pr(x ≤ M ) 6= 0.5 Let the test statistic be: tn = n X I(xi ≤ M ) i=1 This test statistic has a simple interpretation: it is the number of observations in the sample less than our candidate median. What is the exact distribution of tn under the null? 1 ECON 835, Fall 2012 2 b) Suppose n = 100. Calculate the critical values at which you would reject the null at 5% significance. You can use just about any statistical package to calculate percentiles of the binomial distribution, or you could use the Excel function CRITBINOM. c) Use the previous results to construct a formula for the exact 95% confidence interval for the median when n = 100. 3 The joint normal distribution We say that the k-vector x has a joint normal distribution with mean µ and covariance matrix Σ (or x ∼ N (µ, Σ)) if its PDF takes the form: 1 1 0 −1 (x − µ) Σ (x − µ) exp f (x) = − 2 (2π)k/2 |Σ|1/2 The joint normal distribution has several features that make it useful: 1. If x ∼ N (µ, Σ) then Ax + b ∼ N (Aµ + b, AΣA0 ) for any j-vector b and j-by-k matrix A. 2. Let x1 , x2 , . . . xk be a sequence of k independent N (0, 1) random variables, and let x = [x1 x2 · · · xk ]. Then x ∼ N (0, I) where I is the k-by-k identity matrix. 3. Suppose x can be partitioned into two submatrices x1 and x2 , where x1 is length j and x2 is length k, and that: x1 µ1 Σ11 Σ12 x= ∼N , x2 µ2 Σ21 Σ22 Then: −1 x1 |x2 ∼ N µ1 + Σ12 Σ−1 22 (x2 − µ2 ), Σ11 − Σ12 Σ22 Σ21 Suppose that Dn = x1 , x2 , . . . xn is a random sample of size n independent N (µ, σ 2 ) random variables. We said in class that under the null hypothesis H0 : µ = µ0 , the test statistic: x ¯−µ0 √ √ x ¯ − µ0 σ/ n t= n = s s/σ had the Student’s t distribution with n − 1 degrees of freedom. Now “the Student’s t distribution with d degrees of freedom” is defined as the distribution of the ratio a/b where a and b are independent random variables, a ∼ N (0, 1) and b is the square root of a χ2 (d) random variable. We won’t go through the full proof that our test statistic has the tn−1 distribution ,but we will prove a few of the pieces. Pn x ¯−µ √ ∼ N (0, 1). a) Let x ¯ = n1 i=1 xi . Prove1 that x ¯ ∼ N (µ, σ 2 /n) and that σ/ n Pn 1 b) Prove that xi − x ¯ is normal and independent of x ¯ for all i. Does this imply that s2 = 1−n ¯)2 i=1 (xi − x √ 2 and s/σ = s /σ are independent of x ¯? 1 The hard/important part here is not showing that E(¯ x) = µ or that var(¯ x) = σ 2 /n, it’s showing that x ¯ is normally distributed. ECON 835, Fall 2012 4 3 More on the joint normal distribution a) Suppose that y x ∼N µy µx 2 σy , σxy σxy σx2 where y and x are scalar random variables. Find the conditional distribution of y given x. You can express this distribution by a PDF or simply in the same sort of way I’m describing their joint distribution. b) We will use a simple counterexample to prove that two normal random variables are not necessarily jointly normal. Let x ∼ N (0, 1) and let x if |x| ≤ 1 y= −x if |x| > 1 Prove that y ∼ N (0, 1) but that x and y are not jointly normal.