University of Illinois Spring 2015 ECE 561: Problem Set 1: Solutions Statistical Decision-Making Framework, Binary Bayesian and Minimax Hypothesis Testing 1. [Statistical Decision Making]. (a) We may compute the conditional risks for the eight possible decision rules as follows. δ δ1 δ2 δ3 δ4 δ5 δ6 δ7 δ8 1 0 0 0 0 1 1 1 1 2 0 0 1 1 0 0 1 1 3 0 1 0 1 0 1 0 1 R0 0 0 0.6 0.6 0.4 0.4 1 1 R1 10 5 7.5 2.5 7.5 2.5 5 0 0.5(R0 + R1 ) 5 2.5 4.05 1.55 3.95 1.45 3 0.5 max(R0 , R1 ) 10 5 7.5 2.5 7.5 2.5 5 1 It is clear that there is no rule that can be better than both δ2 and δ8 , and hence there is no best rule based on conditional risks alone. (b) It follows from the table that δ8 (always choose 1) is the best Bayes as well as minimax rule in D. ˜ (c) Randomization does not improve minimum Bayes risk, hence δ8 is still the best rule in D. However it is easy to check that the rule δ2 δ˜m = δ8 with prob. 1/6 . with prob. 5/6 has a minimax risk of 5/6, which is better than the minimax risk of the nonrandomized minimax rule from part (b). Since R0 (δi ) is smaller than R0 (δ8 ) for any i ∈ {3, 4, 5, 6}, one can also randomize δ8 with δi , i ∈ {3, 4, 5, 6} to achieve a smaller minimax error. 2. [Optimal Rule Based on Conditional Risks] Let δ0 be the decision rule that always chooses hypothesis 0. It is easy to check that R0 (δ0 ) = C0,0 . Similarly, let δ1 be the decision rule that always chooses hypothesis 1: R1 (δ1 ) = C1,1 . Suppose there exists a “best” rule δ that divides the observation space Y into Y0 and Y1 . Since δ is a best rule, we have R0 (δ) ≤ R0 (δ0 ) ⇒ C0,0 P0 (Y0 ) + C1,0 P0 (Y1 ) ≤ C0,0 ⇒ C1,0 P0 (Y1 ) ≤ C0,0 P0 (Y1 ) Since C0,0 < C1,0 , the inequality above is possible only if P0 (Y1 ) = 0. Similarly, by exploiting the fact that R1 (δ) ≤ R1 (δ1 ), we get P1 (Y0 ) = 0. Thus, we proved that pi (y) has non-zero support only on Yi , i = 0, 1. Since Y0 and Y1 are disjoint, this implies that p0 (y) and p1 (y) have disjoint supports. 3. [Health Insurance] Since A = {buy, not} and Y = {0, 1}, there are four deterministic decision rules. δ δ1 δ2 δ3 δ4 0 buy buy not not 1 buy not buy not R0 (δ) 1 11/2 11/2 10 R1 (δ) 1 0 1 0 0.2R0 (δ) + 0.8R1 (δ) 1 1.1 1.9 2 max{R0 (δ), R1 (δ)} 1 11/2 11/2 10 where R0 (δ) = E0 [C(δ(Y ), 0)] = X p0 (y)C(δ(y), 0) = y=0,1 R1 (δ) = E1 [C(δ(Y ), 1)] = X 1 [C(δ(0), 0) + C(δ(1), 0)] 2 p1 (y)C(δ(y), 1) = C(δ(1), 1) y=0,1 Listing all four decision rules in above table, we observe that δ1 (y) = buy ∀y, is the minimax rule among all deterministic decision rules. By observing that R0 (δ1 ) < R0 (δi ), i = 2, 3, 4, it is clear that ˜ < 1. As a result, δ1 is also the minimax rule among all there is no randomized rule δ˜ such that R0 (δ) randomized rules. 4. [Binary Communication with Erasures] (a) The Bayes rule chooses the decision with smallest a posteriori cost given Y = y. Now, C(d|y) = 1 X C(d, j)π(j|y). j=0 Thus C(0|y) = π(1|y), C(1|y) = π(0|y), and C(e|y) = c[π(0|y) + π(1|y)] = c, with π(0|y) = p0 (y)π0 p0 (y)π0 1 = = p(y) p0 (y)π0 + p1 (y)π1 1 + exp(2y/σ 2 ) and π(1|y) = 1 − π(0|y) = 1 . 1 + exp(−2y/σ 2 ) We erase (i.e., choose d = e) if c < min{π(0|y), π(1|y)}. Since min{π(0|y), π(1|y)} is always less that 0.5, we never erase if c ≥ 0.5. If c < 0.5, we consider two cases: (i) if y ≥ 0, then min{π(0|y), π(1|y)} = π(0|y) and hence we erase 1 σ2 1 if c < i.e., if y < ln − 1 =: t 1 + exp(2y/σ 2 ) 2 c (ii) if y < 0, then min{π(0|y), π(1|y)} = π(1|y) and hence we erase 1 σ2 1 if c < i.e., if y > − ln − 1 = −t 1 + exp(−2y/σ 2 ) 2 c Thus Ye = (−t, t). We choose 0 if π0 (y) ≤ min{c, π(1|y)}. From above it is clear that Y0 = (−∞, −t]. 2 Finally, we choose 1 if π1 (y) ≤ min{c, π(0|y)}. Thus Y1 = [t, ∞). To summarize, 0 e δB (y) = 1 y ≤ −t −t < y < t , y≥t σ2 ln where t = 2 1 −1 , c if c < 0.5, (b) Recall that we erase if c < min{π(0|y), π(1|y)}. Since min{π(0|y), π(1|y)} is always less that 0.5, we never erase if c ≥ 0.5. 0 y≤0 δB (y) = , if c ≥ 0.5. 1 y>0 5. [Minimum Bayes Risk Curve and Minimax Rule] For binary hypothesis testing, the structure of the Bayes rule is given by ( (y) ≥η 0 L(y) , pp01 (y) , δB (y) = 1 L(y) < η where η, π0 (C10 − C00 ) π0 , = C01 − C11 10(1 − π0 ) and η is monotonically increasing in π0 . It is straightforward to see that 5/8 5/12 L(y) = ∞ When π0 10(1−π0 ) ≤ 5/12, which is equivalent to π0 ≤ 25 31 , δB (y) = δ8 = 1 When 5 12 < π0 10(1−π0 ) ≤ 58 , which is equivalent to 25 31 δB (y) = δ6 = When 5 8 < π0 10(1−π0 ) , which is equivalent to 25 29 y=1 y=2 . y=3 the Bayes rule in this case is for any y. < π0 ≤ 1 0 25 29 , the Bayes rule in this case is y = 1, 3 . y=2 < π0 ≤ 1, the Bayes rule in this case is δB (y) = δ2 = 1 0 Now consider V (π0 ) = min r(π0 , δ). It follows that for δ y=3 . y = 1, 2 π0 10(1−π0 ) ≤ 5/12, V (π0 ) = r(π0 , δ8 ) = π0 ; and for 5 12 < π0 10(1−π0 ) ≤ 58 , V (π0 ) = r(π0 , δ6 ) = 2.5 − 2.1π0 ; 3 5 4.5 4 3.5 3 2.5 δ6 δ2 2 1.5 minimax rule 1 δ8 0.5 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 25/31 Figure 1: V (π0 ) and for 5 8 < π0 10(1−π0 ) , V (π0 ) = r(π0 , δ2 ) = 5 − 5π0 . The risk function V (π0 ) as a function of π0 is shown in the Figure 1. 25 . Since V (π0 ) is non-differentiable at π0∗ , The maximizer of V (π0 ) can be easily computed to be π0∗ = 31 there does not exist a deterministic minimax rule. The minimax rule is constructed by randomizing between δ8 and δ6 , which is also an equalizer rule. Let δ8 w.p. ν δ˜ = . δ6 w.p. 1 − ν ˜ and R1 (δ), ˜ we solve for ν as follows Equalizing R0 (δ) νR0 (δ8 ) + (1 − ν)R0 (δ6 ) = νR1 (δ8 ) + (1 − ν)R1 (δ8 ) ν + 0.4(1 − ν) = 2.5(1 − ν) 21 ν= . 31 So the minimax rule is δ˜ = δ8 w.p. 21 31 δ6 w.p. 10 31 . 6. [Binary Hypothesis Testing] (a) It follows by definition that L(y) = 3y 2 for y ∈ [−1, 1], and η = 1 for uniform costs and equal priors. The Bayes rule is given by ( 1 for |y| ≥ √13 δB (y) = . 0 for |y| < √13 4 The risk associated with the Bayes rule with equal priors is Z √1 Z 1 2 3 3y 1 1 1 r(δB ) = · 2 dy + · 2 dy 1 2 2 2 2 √ 0 3 1 1 = − √ . 2 3 3 (b) The Bayes rule for a given π0 is δB (y) = 1 for |y| ≥ √ 0 for |y| < √ √ π0 3(1−π0 ) √ π0 . (1) 3(1−π0 ) It is known that if there exists a Bayes rule that equalizes the conditional risks under both hypotheses, it is also a minimax rule. As shown by (1), the structure of any Bayes rule is given by 1 for |y| ≥ η δB (y) = . 0 for |y| < η And the corresponding conditional error probabilities can be computed to be Z 1 1 dy = 1 − η, R0 (δB ) = 2 2 η and Z R1 (δB ) = 2 0 η 3 3 y dy = η 3 . 2 Equalizing R0 and R1 , we get that η = 0.6824. So the minimax rule is given by 1 for |y| ≥ 0.6824 δm (y) = . 0 for |y| < 0.6824 And the corresponding risk is R0 = R1 = 0.3177. (We can reach the same answer by arguing that because the distribution of L(y) has no point mass under either distribution, the risk function V (π0 ) is differentiable for all π0 ∈ (0, 1). As a result, the minimax rule is given by the Bayes rule that equalizes the conditional risks under both hypotheses.) 7. [Binary Hypothesis Testing with Nonuniform Costs] (a) The likelihood ratio L(y) is given by L(y) = 2e−|y| . With equal priors (π0 = π1 = 0.5), the threshold for Bayes rule is given by τ= C1,0 − C0,0 1 = . C0,1 − C1,1 4 Therefore, the Bayes rule is δB (y) = 1, 0, if |y| ≤ ln 8 . otherwise (b) The corresponding Bays risk is equal to 1 1 1 C1,0 P0 (Y1 ) + C0,1 P1 (Y0 ) = 2 2 2 5 7 1 15 + = . 8 32 32
© Copyright 2024