1/20 EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) Marcus Chambers Department of Economics University of Essex 7 November 2013 EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) 2/20 Outline 1 Review 2 OLS in large samples: relaxing normality 3 Instrumental variables estimation Reference: Greene, chapters 4 and 8. EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) Review 3/20 Model: y = Xβ + . OLS estimator, b = (X 0 X)−1 X 0 y, is BLUE under classical assumptions. Large sample concepts: consistency, limiting distribution. MLE: equivalent to OLS in classical model under normality, efficient. Tests of nonlinear restrictions → large sample hypothesis tests. How important is the normality assumption? EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) OLS in large samples: relaxing normality 4/20 Consider the model yi = xi0 β + i ; i |X ∼ IID(0, σ 2 ). NB: ‘IID(0, σ 2 )’ means ‘independently and identically distributed with mean zero and variance σ 2 ’. In this model E(b) = β and var(b|X) = σ 2 (X 0 X)−1 as usual. Under normality i.e. if |X ∼ N(0, σ 2 In ) then b|X ∼ N(β, σ 2 (X 0 X)−1 ). But what can we say about the properties of b in the model without the assumption of normality? EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) OLS in large samples: relaxing normality 5/20 Note that we can write b in the form 1 0 −1 1 0 1 0 −1 1 0 b= XX Xy =β+ XX X . n n n n (1) It is typically assumed that: (a) Well-defined limit: 1 0 X X = Qxx (K × K, positive definite); plim n (b) X uncorrelated with : 1 0 plim X = 0 (K × 1). n EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) OLS in large samples: relaxing normality 6/20 Applying the plim operator to (1) and using Slutsky’s Theorem we obtain X 0 X −1 X0 plim b = plim β + plim plim n n −1 = β + Qxx × 0 = β. Hence b is a consistent estimator of β. What about the large sample distribution? Under the assumptions of the model 1 d √ X 0 → N(0, σ 2 Qxx ). n EC501 Econometric Methods and Applications (2) 5. Large Sample Methods (Continued) OLS in large samples: relaxing normality 7/20 Using (1) we obtain √ n(b − β) = X0X n −1 1 √ X 0 . n From Cramer’s Theorem √ d −1 2 −1 n(b − β) → N(0, σ 2 Q−1 xx Qxx Qxx ) = N(0, σ Qxx ). (3) We can use this result to justify using the normal distribution in large (but finite) samples: (b − β) ∼ N(0, σ 2 n−1 Q−1 xx ) ⇒ b ∼ N(β, σ 2 (X 0 X)−1 ). (4) The above normal distribution is regarded as holding approximately for finite n. EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) Instrumental variables estimation 8/20 Consider the model y = Xβ + , i = IID(0, σ 2 ), i = 1, . . . , n. (5) We have examined the properties of the OLS estimator in this model assuming 1 0 X X = Qxx (nonsingular); plim (6) n 1 0 X = 0. (7) plim n Because b = β + (X 0 X/n)−1 (X 0 /n) we find that plim b = β. EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) Instrumental variables estimation 9/20 But suppose, instead, that elements of X are correlated with in the limit, so that (7) becomes 1 0 X = γ 6= 0. (8) plim n We now have plim b = β + Q−1 xx γ 6= β, and so b is no longer consistent. When might (8) arise? EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) Instrumental variables estimation 10/20 A simple example involves the consumption function in a closed economy: Ci = β1 + β2 Yi + i , Yi = Ci + Ii + Gi = β1 + β2 Yi + i + Ii + Gi , → Yi = 1 (β1 + Ii + Gi + i ) . 1 − β2 Here, Yi , the regressor in the consumption function, is correlated with (in fact, depends on) i , the disturbance. How can we construct a consistent estimator of β? EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) Instrumental variables estimation 11/20 Suppose we can find a set of L instrumental variables, the observations being contained in the n × L matrix Z. We require Z to satisfy the following properties: (a) Well-defined limit: 1 0 Z Z = Qzz (L × L, positive definite); plim n (b) Z correlated with X: 1 0 plim Z X = Qzx (L × K, rank K); n (c) Z uncorrelated with : plim EC501 Econometric Methods and Applications 1 0 Z = 0. n 5. Large Sample Methods (Continued) Instrumental variables estimation 12/20 The IV estimator may be obtained in two steps: 1. Regress each of the variables in X on those in Z and ˆ obtain the matrix of fitted values, X; ˆ 2. Regress y on X to obtain bIV , the IV estimator of β. Let’s look at these two steps in more detail. EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) Instrumental variables estimation 13/20 Step 1: note that X = [x1 , . . . , xK ]. Regressing each xk on Z is equivalent to estimating xk = Zak + uk , k = 1, . . . , K, the estimator for which is ˆ ak = (Z 0 Z)−1 Z 0 xk (L × 1). The fitted values are then ˆxk = Zˆ ak = Z(Z 0 Z)−1 Z 0 xk = PZ xk (n × 1), where PZ = Z(Z 0 Z)−1 Z 0 is the projection matrix for Z. Combining the fitted values in a matrix yields ˆ = [ˆx1 , . . . , ˆxK ] = [PZ x1 , . . . , PZ xK ] = PZ X. X EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) Instrumental variables estimation 14/20 Step 2: we now estimate the model ˆ + . y = Xβ The estimator of β in this model is the IV estimator: bIV ˆ 0 X) ˆ −1 X ˆ 0y = (X = (X 0 P0Z PZ X)−1 X 0 P0Z y = (X 0 PZ X)−1 X 0 PZ y −1 0 = X 0 Z(Z 0 Z)−1 Z 0 X X Z(Z 0 Z)−1 Z 0 y, (9) where we have used the symmetry (P0Z = PZ ) and idempotency (P2Z = PZ ) of PZ . EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) Instrumental variables estimation 15/20 What are the (asymptotic) properties of bIV ? By substituting y = Xβ + in (9) we obtain bIV −1 0 = β + X 0 Z(Z 0 Z)−1 Z 0 X X Z(Z 0 Z)−1 Z 0 #−1 " X 0 Z Z 0 Z −1 Z 0 X 0 Z Z 0 Z −1 Z 0 X . = β+ n n n n n n Taking the plim we obtain −1 0 −1 plim bIV = β + [Q0zx Q−1 zz Qzx ] Qzx Qzz · 0, so that plim bIV = β and bIV is a consistent estimator of β. EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) Instrumental variables estimation 16/20 Under appropriate conditions ensuring that 1 d √ Z 0 → N(0, σ 2 Qzz ) n we find that, as n → ∞, #−1 " √ X 0 Z Z 0 Z −1 Z 0 X 0 Z Z 0 Z −1 Z 0 X √ n(bIV − β) = n n n n n n d −1 → N 0, σ 2 (Q0zx Q−1 . zz Qzx ) Hence for large (but finite) sample size n we have the approximate distribution ˆ 0 X) ˆ −1 . bIV ∼ N β, σ 2 (X (10) EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) Instrumental variables estimation 17/20 The distribution in (10) cannot be used in practice because σ 2 is unknown. We can estimate σ 2 using σ ˆ2 = ˆ0 ˆ where ˆ = y − XbIV . n−K (11) Note that the appropriate residuals are y − XbIV and not ˆ IV . y − Xb The IVE can be obtained from the minimisation problem bIV = arg min SIV (β) where SIV (β) = (y − Xβ)0 PZ (y − Xβ). β You may wish to verify this! EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) Instrumental variables estimation 18/20 How do we choose instruments? Recall that we require Z to be uncorrelated with and correlated with X. Recall the consumption function example: Ci = β1 + β2 Yi + i , Yi = Ci + Ii + Gi . If Ii and Gi are exogenous then they would be suitable instruments because they are uncorrelated with i but determine Yi (and hence are correlated with Yi ). EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) Instrumental variables estimation 19/20 In the above example, the complete model is known and the choice of instruments is obvious. But often it is infeasible and/or impractical to specify a large set of equations when we are really only interested in one. The general rule is that instruments must be variables that affect yi only through their effect on the regressors for which they act as instruments. EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued) Summary 20/20 Summary OLS in large samples: relaxing normality instrumental variables estimation Next week: generalised linear regression model (E(0 ) 6= σ 2 In ) heteroskedasticity (σ 2 not constant across i) EC501 Econometric Methods and Applications 5. Large Sample Methods (Continued)