4 Solutions, Homework 4 Exercise 4.1. Let X1 , X2 , . . . be i.i.d. with P(Xi = (−1)k k) = C/k 2 log k for k ≥ 2 where C is chosen to make the sum of all probabilities one. Show that E[|Xi |] = ∞ but there is a finite constant µ such that Sn → µ in probability as n → ∞. n Solution: We see that E[|X|] = X k≥2 k× X 1 C = C = ∞. k 2 log k k log k k≥2 Next, we want to check the hypotheses in the Weak Law of Large number, i.e. x lim P(|X| ≥ x) = 0. x→∞ Since X only takes integer values, it is actually enough to check the above limit only over integer values x = n. We have that Z ∞ X C 1 1 1 ≈ C ≤ C . P(|X| ≥ n) = k 2 log k x2 log x log n n n k≥n Therefore, indeed, n lim P(|X| ≥ n) = 0. n→∞ According to the WLLN (the most general form), we now have Sn − µn → 0 in probability as n → ∞, n where µn = E[X1{|X|≤n} ]. Now, we only have to see that, actually µn = n X C(−1)k k=2 k log k , which coverges to the finite sum of the alternate series: µn → µ = ∞ X C(−1)k k=2 Exercise 4.2. k log k ∈ R. 1. Show that if X ≥ 0 is integer valued, then E[X] = P n≥1 P(X ≥ n). 2. find a similar expression for E[X 2 ]. Solution: 1. We know that Z ∞ P(X ≥ x)dx = E[X] = 0 ∞ Z X n=1 beacuse X takes only integer values. 1 n n−1 P(X ≥ x)dx = ∞ X n=1 P(X ≥ n), 2. We could easily write that as 2 E[X ] = ∞ X 2 P(X ≥ n) = n=1 ∞ X P(X ≥ √ n). n=1 However, an even better way to do this is 2 Z ∞ 2xP(X ≥ x)dx = E[X ] = 0 ∞ Z X E[X 2 ] = 2xP(X ≥ x)dx = n−1 n=1 leading to n ∞ X ∞ X Z n P(X ≥ n) n=1 2xdx n−1 (2n + 1)P(X ≥ n). n=1 Either of the above representations will be considered correct for grading. Rx Exercise 4.3. If h(y) ≥ 0 and H(x) = −∞ h(y)dy, show that Z ∞ h(y)P(X > y)dy E[H(X)] = −∞ Solution: This is just a usual application of Fubini. To be precise, Z ∞ Z ∞ Z ∞ h(y)P(X ≥ y)dy. h(y)E[1{y≤X} ] = 1{y≤X} h(y)dy = E[H(X)] = E −∞ −∞ −∞ Remark: A Weak Law of Large Numbers, in an extended sense, is a deterministic description (using convergence in probability) of the empirical average Sn /n for large sample size n. Usually we think about it as either or Sn → a ∈ [−∞, ∞], n (4.1) Sn bn → +(−)1 for some → ∞. bn n (4.2) When X ∈ L1 we know that we have the weak law in (4.1) with a = E[X] ∈ (−∞, ∞). We only try to find a weak law in the form (4.2) when X is not integrable, so, because Sn takes large values, we try to normalize it by something larger than n to obtain (4.2). Relation (4.2) implies (4.1) since Sn S n bn = → +(−)1 · ∞ = +(−)∞, n bn n so (4.2) (when it holds) provides better information about the empirical average for large samples than (4.1) with a = +(−)∞. Exercise 4.4. (Weak Law for Positive random variables, in the form (4.1)) Using truncation, show that if X ≥ 0 and E[X] = ∞, then Sn → ∞, in probability as n → ∞. n 2 Solution: If we truncate (each of the i.i.d random variables) at level M the empirical average S˜n /n of the truncations converges to E[X1{X≤M } ]. Now, because each Xi is positive, truncation decreases the sum, so S˜n Sn ≥ → E[X1{X≤M } ], in probability. n n This is true for each M , so we can let M % ∞. By monontone convergence we have E[X1{X≤M } ] % E[X] = ∞. From here we obtain the conclusion, but the full argument involves a precise definition of what it means for a sequence to converge in probability to infinity. Exercise 4.5. (Weak Law for positive random variables in form (4.2)) Assume X ≥ 0. Rs 1. Assume E[X] = ∞ and lims→∞ s(1 − FX (s)) = 0. If we denote by µ(s) = 0 xdFX (x), then Sn → 1, in probability as n → ∞, nµ(n) and µ(n) → ∞. Please note, as in the short discussion above, that this is in line with the conclusion of the previous exercise. 2. Assume E[X] = ∞ and ν(s) = µ(s) → ∞, as s → ∞. s(1 − FX (s)) Show that for n large we can choose bn such that nµ(bn ) = bn and Sn → 1, in probability as n → ∞, bn and bn /n → ∞. Please note that the assumptions of item 1 are stronger than the assumptions of item 2. In particular, if assumptions of item 1 hold, then bn /nµ(n) → 1. Solution: 1. Actually, the hypothesis lims→∞ s(1 − FX (s)) = 0 is exactly what is needed to apply the WLLN (the most general version we learned), because 1 − F (s) = P(X > s). We then have Sn − µ(n) → 0, n but µ(n) % ∞, so we can obtain the result by dividing the limit above by µ(n). 2. We note that Z s µ(s) x = dF (x) → 0, as s → ∞, s 0 s by Dominated Convergence. The function s→ µ(s) s is Right Continuous. In case it has jumps, the jumps equal to the jumps of F . In particular, such jumps are positive jumps. We can, therefore, find for large n a bn such that µ(bn )/bn = 1/n 3 and bn → ∞. This is not entirely obvious, but you are certainly not expected to make a full argument here, the idea is enough. Since bn → ∞, we can conclude that µ(bn ) → E[X] = ∞, by monotone convergence. Therefore, bn /n = µ(bn ) → ∞ as well. We now apply the weak law for triangular arrays (page 41 in the textbook). The hypotheses there are satisfied, since (a) n(1 − F (bn )) = n µ(bn ) 1 1 = →0 bn ν(bn ) ν(bn ) (b) n X ˜2 ] E[X n,k k=1 b2n n = 2 bn Z 0 bn n 2y(1−F (y))dy = 2 bn Z 0 bn µ(y) nµ(bn ) dy ≤ ν(y) b2n Z 0 bn 1 dy → 0. ν(y) Now, from the triangular array WLLN, we have (Sn − an )/bn → 0, where actually an = nµ(bn ) = bn . Exercise 4.6. (St. Petersburg paradox) Consider X distributed as P(X = 2j ) = 2−j , j = 1, 2, . . . . In other words X is the outcome of a game where a fair coin is tossed and if heads comes up first time after j tosses, then the player receives 2j dollars. Let Sn = X1 + · · · + Xn where Xi are i.i.d. distributed as X. Use item 2 in the previous problem to show that Sn → 1, in probability as n → ∞. n log2 (n) This is called a “paradox” since, although this game has an infinite value, it seems that someone wouldn’t pay too much (more than 40 dollars, argued historically) to play this game (since the probability of gains above 40 is very small). This problem shows that, if someone plays this game and pays log2 (n) per game, it takes around n games to break even. However, if log2 (n) = 40, then n = 240 is very large. Solution: we just apply the second part of the previous problem. We actually need to check the limitsnot over s → ∞ but only over integers of the form n = 2j , because P −j all probability mass lies at these points. Since 1 − F (2j ) = = 2−j , we have k>j 2 2j (1 − F (2−j ) = 1. Since µ(2j ) → E[X] = ∞ the hypotheses of the previous problem, second part, are satisfied.P Now, µ(2j ) = jk=1 2k ×2−k = j and is flat in between [2j , 2j+1 ). This means that solving for bn such that nµ(bn ) = bn amounts to finding j with the property nj ∈ [2j , 2j+1 ). then set bn = nj. The j is found by solving for each n for j such that log2 n + log2 j ∈ [j, j + 1). This does have a solution j(n) % ∞. Dividing by j(n), we obtain log2 n → 1, j(n) wich translates easily as bn nj(n) = → 1 as n → ∞. n log2 n n log2 n The proof is now complete, taking into account the second part of the previous problem. 4