1. Sample Space and Probability Part I ECE 302 Fall 2009 TR 3‐4:15pm Purdue University, School of ECE Prof. Ilya Pollak ProbabilisJc Model •  Sample space Ω = the set of all possible outcomes of an experiment –  E.g., {+$1, ‐$1} for a one‐month stock movement in the opJon example; {$102, $100, $98} for the stock value aUer two months; {Obama, McCain, neither} for a voter’s preference, etc. •  Probability law which assigns to a set A of possible outcomes (also called an event) a number P(A), called the probability of A. Sets: basic terms and notaJon, 1 A countably infinite (or countable) set is a set with infinitely
many elements which can be enumerated in a list, e.g.,
the set of all integers {0,−1,1,−2,2,…}
An example of an uncountable set is the set of all real numbers
between 0 and 1, denoted [0,1]
Sets: basic terms and notaJon, 2 The set of all x that have a certain property P is denoted by
{ x | x satisfies P},
e.g., the interval [0,1] can alternatively be written as {x | 0 ≤ x ≤ 1}.
Set operaJons: complement Set operaJons: union More generally,
∞
S
n=1
n
= S1 ∪ S2 ∪… = { x | x ∈ Sn for some n}
Set operaJons: intersecJon More generally,
∞
S
n=1
n
= S1 ∩ S2 ∩… = { x | x ∈ Sn for all n}
Disjoint sets ParJJon CollecJvely exhausJve sets Example: three coin tosses H
H
T
T
H
T
H
HHH
T
HHT
H
HTH
T
HTT
H
THH
T
THT
H
TTH
T
TTT
Sample space Ω = {HHH,HHT,HTH,HTT,THH,THT,TTH,TTT}
Event "heads on the first and second toss" is the set {HHH,HHT}
Example: three coin tosses Example: three coin tosses Example: three coin tosses Example: three coin tosses Example: three coin tosses Example: three coin tosses Example: presidenJal elecJon % for Obama
100%
100%
% for McCain
Example: presidenJal elecJon % for Obama
100%
100%
% for McCain
Note: Even though the sample space is discrete in reality
(the quantum is 1 voter out of 130M, or 0.00000077%),
it is more conveniently modeled as continuous.
Example: presidenJal elecJon % for Obama
100%
100%
% for McCain
Is this sample space enough for answering these questions:
(1) Will Obama win the popular vote?
(2) Will Obama win the election?
Example: presidenJal elecJon % for Obama
100%
100%
% for McCain
Is this sample space enough for answering these questions:
(1) Will Obama win the popular vote?
(2) Will Obama win the election?
(1) No: need to include 3rd-party candidates
(2) No: need both 3rd-party candidates and electoral
(not popular) vote
Example: presidenJal elecJon % for Obama
100%
100%
% for McCain
Example: presidenJal elecJon % for Obama
100%
100%
% for McCain
€
ProbabilisJc Model •  Sample space Ω •  Probability law: assigns to an event A a number P(A) saJsfying the following probability axioms: 1. P(A) ≥ 0
2. If A ∩ B = ∅, then P(A ∪ B) = P(A) + P(B)
∞  ∞
If A1, A2 ,… are disjoint, then P An  = ∑ P(An )
 n=1  n=1
3. P(Ω) = 1
RelaJonship between probability and relaJve frequency of occurrence Example: three coin tosses Suppose all outcomes are equally likely
P(Ω) = 1 by axiom 3
P ({HHH}) + P ({HHT}) + … + P ({TTT}) = P(Ω) by axiom 2
P ({HHH}) = P ({HHT}) = … = P ({TTT}) = 1/8
P(S4 ) = P("two tails in a row") = P ({HTT,TTH,TTT}) = 3/8 by axiom 2
Discrete uniform probability law : if Ω consists of N equally likely
outcomes, then, for any event A,
number of elements in A
P(A) =
N
Example: presidenJal elecJon % for Obama
100%
100%
% for McCain
Example: presidenJal elecJon % for Obama
100%
100%
% for McCain
Example: some properJes of
probability laws Example: some properJes of
probability laws Example: some properJes of
probability laws Example: some properJes of
probability laws Example: some properJes of
probability laws Example: some properJes of
probability laws (a) If A ⊂ B then P(A) ≤ P(B)
Proof : Let C = A c ∩ B
Then B = A ∪ C
and C ∩ A = ∅ (since C ⊂ A c ).
Therefore,
Axiom 2
P(B) = P(A ∪ C) = P(A) + P(C)
€
Axiom 1 applied to P (C )
≥
P(A)
Example: some properJes of
probability laws Example: some properJes of
probability laws Example: some properJes of
probability laws Example: some properJes of
probability laws Example: some properJes of
probability laws Example: some properJes of
probability laws €
Example: some properJes of
probability laws (c) P(A ∪ B) ≤ P(A) + P(B)
Proof :
Use property (b) : P(A ∪ B) = P(A) + P(B) − P(A
∩
B) ≤ P(A) + P(B)


≥0, by Axiom 1
Example: some properJes of
probability laws (d) P(A ∪ B ∪ C) = P(A) + P(A c ∩ B) + P(A c ∩ B c ∩ C)
Proof :
Since A, A c ∩ B, A c ∩ B c ∩ C form a partition for A ∪ B ∪ C,
the statement follows from Axiom 2.
€
CondiJonal Probability •  NotaJon: P(A|B) •  Probability of event A, given that event B occurred •  DeﬁniJon: assuming P(B)≠0, •  CondiJonal probabiliJes specify a probability law
on the new universe B (exercise) Example: Problem of Points •  Best two out of three fair coin ﬂips •  Helen bets on H, Tom bets on T (a)  What’s the probability that Helen wins 1st round? (b)  What’s the probability that Helen wins overall? (c)  The game is interrupted aUer Helen wins 1st round.
What’s the condiJonal probability that she would
have won overall? •  SoluJon (a)  ½ ‐‐ follows directly from the fact that the coin is fair. Problem of Points, SoluJon H
HHH
T
HHT
H
HTH
T
HTT
H
THH
T
THT
T
H
TTH
P(H wins 1st round) = 1/2
T
TTT
H
H
T
T
H
H wins 1st round
H wins overall
P(H wins 1st round and H wins overall) = 3/8
P(H wins overall | H wins 1st round)
= P(H wins 1st round and H wins overall)/ P(H wins 1st round)
= (3/8)/(1/2) = 3/4
AlternaJvely,… View the blue set as the new sample
space. Three out of four equally likely
outcomes result in H’s overall win.
Therefore,
P(H wins overall | H wins 1st round) = 3/4.
HHH
H wins 1st round
HHT
H wins overall
HTH
HTT
Example 1.9: Intrusion DetecJon Event A: intrusion
Event B: alarm
Suppose that, from past experiences, we know that
P(A) = 0.02,
P(B|Ac) = 0.05,
P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm
probability P(B and Ac)
Example: Intrusion DetecJon Event B: alarm
Event A: intrusion
Suppose that, from past experiences, we know that
P(A) = 0.02,
P(B|Ac) = 0.05,
P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm
probability P(B and Ac)
Bc∩A
A
B∩A
B∩Ac
Ac
Bc∩Ac
Example: Intrusion DetecJon Event B: alarm
Event A: intrusion
Suppose that, from past experiences, we know that
P(A) = 0.02,
P(B|Ac) = 0.05,
P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm
probability P(B and Ac)
P(Bc|A)=0.01
P(A) = 0.02
Bc∩A = missed detection
A
B∩A
P(B|Ac)=0.05
Ac
B∩Ac = false alarm
Bc∩Ac
Example: Intrusion DetecJon Event B: alarm
Event A: intrusion
Suppose that, from past experiences, we know that
P(A) = 0.02,
P(B|Ac) = 0.05,
P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm
probability P(B and Ac)
P(Bc|A)=0.01
P(A) = 0.02
Bc∩A = missed detection
A
B∩A
P(B|Ac)=0.05
Ac
B∩Ac = false alarm
Bc∩Ac
P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
Example: Intrusion DetecJon Event B: alarm
Event A: intrusion
Suppose that, from past experiences, we know that
P(A) = 0.02,
P(B|Ac) = 0.05,
P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm
probability P(B and Ac)
P(Bc|A)=0.01
P(A) = 0.02
Bc∩A = missed detection
A
B∩A
P(B|Ac)=0.05
P(Ac) = 0.98
Ac
B∩Ac = false alarm
Bc∩Ac
P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
Example: Intrusion DetecJon Event B: alarm
Event A: intrusion
Suppose that, from past experiences, we know that
P(A) = 0.02,
P(B|Ac) = 0.05,
P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm
probability P(B and Ac)
P(Bc|A)=0.01
P(A) = 0.02
A
P(B|A)=0.99
P(B|Ac)=0.05
P(Ac) = 0.98
Bc∩A = missed detection
Ac
P(Bc|Ac)=0.95
B∩A
B∩Ac = false alarm
Bc∩Ac
P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
Example: Intrusion DetecJon Event B: alarm
Event A: intrusion
Suppose that, from past experiences, we know that
P(A) = 0.02,
P(B|Ac) = 0.05,
P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm
probability P(B and Ac)
P(Bc|A)=0.01
P(A) = 0.02
A
P(B|A)=0.99
P(B|Ac)=0.05
P(Ac) = 0.98
Bc∩A = missed detection
Ac
P(Bc|Ac)=0.95
B∩A
B∩Ac = false alarm
Bc∩Ac
P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
P(B∩A) = P(A)P(B|A) = 0.02·0.99 = 0.0198
Example: Intrusion DetecJon Event B: alarm
Event A: intrusion
Suppose that, from past experiences, we know that
P(A) = 0.02,
P(B|Ac) = 0.05,
P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm
probability P(B and Ac)
P(Bc|A)=0.01
P(A) = 0.02
A
P(B|A)=0.99
P(B|Ac)=0.05
P(Ac) = 0.98
Bc∩A = missed detection
Ac
P(Bc|Ac)=0.95
B∩A
B∩Ac = false alarm
Bc∩Ac
P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
P(B∩A) = P(A)P(B|A) = 0.02·0.99 = 0.0198
P(B∩Ac) = P(Ac)P(B|Ac) = 0.98·0.05 = 0.049
Example: Intrusion DetecJon Event B: alarm
Event A: intrusion
Suppose that, from past experiences, we know that
P(A) = 0.02,
P(B|Ac) = 0.05,
P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm
probability P(B and Ac)
P(Bc|A)=0.01
P(A) = 0.02
A
Event B
P(B|A)=0.99
P(B|Ac)=0.05
P(Ac) = 0.98
Bc∩A = missed detection
Ac
P(Bc|Ac)=0.95
B∩A
B∩Ac = false alarm
Bc∩Ac
P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
P(B∩A) = P(A)P(B|A) = 0.02·0.99 = 0.0198
P(B∩Ac) = P(Ac)P(B|Ac) = 0.98·0.05 = 0.049
Example: Intrusion DetecJon Event B: alarm
Event A: intrusion
Suppose that, from past experiences, we know that
P(A) = 0.02,
P(B|Ac) = 0.05,
P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm
probability P(B and Ac)
P(Bc|A)=0.01
P(A) = 0.02
A
Event B
P(B|A)=0.99
P(B|Ac)=0.05
P(Ac) = 0.98
Bc∩A = missed detection
Ac
P(Bc|Ac)=0.95
B∩A
B∩Ac = false alarm
Bc∩Ac
P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
P(B∩A) = P(A)P(B|A) = 0.02·0.99 = 0.0198
P(B∩Ac) = P(Ac)P(B|Ac) = 0.98·0.05 = 0.049
P(B) = P(B∩A) + P(B∩Ac) = 0.0198 + 0.049 = 0.0688
Example: Intrusion DetecJon Event B: alarm
Event A: intrusion
Suppose that, from past experiences, we know that
P(A) = 0.02,
P(B|Ac) = 0.05,
P(Bc|A) = 0.01
Find P(B), P(A|B), the missed detection probability P(Bc and A) , the false alarm
probability P(B and Ac)
P(Bc|A)=0.01
P(A) = 0.02
A
Event B
P(B|A)=0.99
P(B|Ac)=0.05
P(Ac) = 0.98
Bc∩A = missed detection
Ac
P(Bc|Ac)=0.95
B∩A
B∩Ac = false alarm
Bc∩Ac
P(Bc∩A) = P(A)P(Bc|A) = 0.02·0.01 = 0.0002
P(B∩A) = P(A)P(B|A) = 0.02·0.99 = 0.0198
P(B∩Ac) = P(Ac)P(B|Ac) = 0.98·0.05 = 0.049
P(B) = P(B∩A) + P(B∩Ac) = 0.0198 + 0.049 = 0.0688
P(A|B) = P(A∩B)/P(B) =0.0198/0.0688 ≈ 0.288
Example 1.18: False‐PosiJve Puzzle • 
• 
• 
• 
• 
A crime commiked on an island, populaJon 5000 A priori, all are equally likely to have commiked it Based on a forensic test, a suspect is arrested Apart from the test, no other evidence Accuracy of the test is 99.9%, i.e., P(test is posiJve | suspect is guilty) = 0.999 and P(test is negaJve | suspect is innocent) = 0.999 •  You are on the jury. Do you have “reasonable doubt”? False‐PosiJve Puzzle: SoluJon •  Proceed similar to the intruder example •  P(+) = P(+ and guilty) + P(+ and innocent) = P(+ | guilty) P(guilty) + P(+ | innocent) P(innocent) = 0.999∙0.0002 + 0.001∙0.9998 = 0.0011996 •  P(guilty | +) = P(+ and guilty) / P(+) = 0.999∙0.0002/0.0011996 ≈ 0.167 !!! •  The seemingly reliable test is not very reliable at all! False‐PosiJve Puzzle: Some IntuiJon •  Suppose this is repeated on 1000 islands •  Suppose we test all 5,000,000 people on 1000 islands False‐PosiJve Puzzle: Some IntuiJon •  Suppose this is repeated on 1000 islands •  Suppose we test all 5,000,000 people on 1000 islands •  On average, we expect roughly the following test
results: ~1 tests 5,000,000
people
1000 guilty
4,999,000
innocent
~999 test +
False‐PosiJve Puzzle: Some IntuiJon •  Suppose this is repeated on 1000 islands •  Suppose we test all 5,000,000 people on 1000 islands •  On average, we expect roughly the following test
results: ~1 tests 5,000,000
people
1000 guilty
4,999,000
innocent
~999 test +
~4999 test +
~4,994,001
test -
False‐PosiJve Puzzle: Some IntuiJon •  Suppose this is repeated on 1000 islands •  Suppose we test all 5,000,000 people on 1000 islands •  On average, we expect roughly the following test
results: ~1 tests 5,000,000
people
1000 guilty
4,999,000
innocent
~999 test +
~4999 test +
~4,994,001
test -
~999 criminals out
of ~5998 who tested +
i.e., roughly 1/6
False‐PosiJve Puzzle: Some IntuiJon •  Suppose this is repeated on 1000 islands •  Suppose we test all 5,000,000 people on 1000 islands •  On average, we expect roughly the following test
results: ~1 tests 5,000,000
people
1000 guilty
4,999,000
innocent
~999 test +
~4999 test +
~999 criminals out
of ~5998 who tested +
i.e., roughly 1/6
~4,994,001
test -
•  I.e., there are so few criminals that the bulk of people
who test posiJve are innocent! Total Probability Theorem A1 A2 B A3 Total Probability Theorem A1 A2 B A3 •  One way of compuJng P(B): P(B) = P(B∩A1) + P(B∩A2) + P(B∩A3) = P(B|A1)P(A1) + P(B|A2)P(A2) + P(B|A3)P(A3) Total Probability Theorem A1 A2 B A3 •  One way of compuJng P(B): P(B) = P(B∩A1) + P(B∩A2) + P(B∩A3) = P(B|A1)P(A1) + P(B|A2)P(A2) + P(B|A3)P(A3) •  More generally, P(B) = Σi P(B|Ai)P(Ai) if Ai’s are mutually exclusive
and B is a subset of the union of Ai’s Bayes’ Rule •  Prior model: probabiliJes P(Ai) Bayes’ Rule •  Prior model: probabiliJes P(Ai) •  Measurement model: P(B|Ai) –  condiJonal probability to observe data B given that
the truth is Ai Bayes’ Rule •  Prior model: probabiliJes P(Ai) •  Measurement model: P(B|Ai) –  condiJonal probability to observe data B given that
the truth is Ai •  Want to compute posterior probabiliBes P(Ai|B) –  CondiJonal probability that the truth is Ai given
that we observed data B Bayes’ Rule •  Prior model: probabiliJes P(Ai) •  Measurement model: P(B|Ai) –  condiJonal probability to observe data B given that
the truth is Ai •  Want to compute posterior probabiliBes P(Ai|B) –  CondiJonal probability that the truth is Ai given
that we observed data B P(Ai ∩ B) P(B | Ai )P(Ai )
P(Ai | B) =
=
P(B)
P(B)
P(B | Ai )P(Ai )
=
(by total probability thm)
∑ P(B | A j )P(A j )
j
MulJplicaJon Rule  n−1 
n 
P  Ai  = P(A1 )P(A2 | A1 )P(A3 | A1 ∩ A2 ) ⋅…⋅ P An  Ai ,
 i=1 
 i=1 
provided all the conditioning events have nonzero probability
Example 1.10 •  Three cards are drawn from a deck of 52 cards, without
replacement. At each step, each one of the remaining cards is
equally likely to be picked. What’s the probability that none of the
three cards is a heart? •  Let Ai={i‐th card is not a heart}, i=1,2,3 Example •  Three cards are drawn from a deck of 52 cards, without
replacement. At each step, each one of the remaining cards is
equally likely to be picked. What’s the probability that none of the
three cards is a heart? •  Let Ai={i‐th card is not a heart}, i=1,2,3 P(A1 ) =
€
€
P(A1c ) =
€
39
52
13
52
€
A1
A1c
Example •  Three cards are drawn from a deck of 52 cards, without
replacement. At each step, each one of the remaining cards is
equally likely to be picked. What’s the probability that none of the
three cards is a heart? •  Let Ai={i‐th card is not a heart}, i=1,2,3 38
P(A2 | A1 ) =
51
P(A1 ) =
39
52
A1
€
€
€
€
P(A1c ) =
€
13
52
€
A1 ∩ A2
13
51
A1c
€
€
A1 ∩ A2c
Example •  Three cards are drawn from a deck of 52 cards, without
replacement. At each step, each one of the remaining cards is
equally likely to be picked. What’s the probability that none of the
three cards is a heart? •  Let Ai={i‐th card is not a heart}, i=1,2,3 P(A3 | A1 ∩ A2 ) =
38
P(A2 | A1 ) =
51
P(A1 ) =
39
52
A1
€
€
13
P(A ) =
52
c
1
€
€
13
51
c
1
€
€
€
€
A1 ∩ A2c
€
A
A1 ∩ A2 ∩ A3
A1 ∩ A2
13
50
€
€
37
50
€
A1 ∩ A2 ∩ A3c
Example •  Three cards are drawn from a deck of 52 cards, without
replacement. At each step, each one of the remaining cards is
equally likely to be picked. What’s the probability that none of the
three cards is a heart? •  Let Ai={i‐th card is not a heart}, i=1,2,3 P(A3 | A1 ∩ A2 ) =
38
P(A2 | A1 ) =
51
P(A1 ) =
39
52
A1
€
€
13
P(A ) =
52
c
1
€
13
51
c
1
€
€
€
€
A1 ∩ A2 ∩ A3c
A1 ∩ A2c
€
A
A1 ∩ A2 ∩ A3
A1 ∩ A2
13
50
€
€
37
50
€
39 38 37
P(A1 ∩ A2 ∩ A3 ) = P(A1 )P(A2 | A1 )P(A3 | A1 ∩ A2 ) =
⋅
⋅
≈ 0.41
52
51
50
€
Example: Prisoner’s Dilemma (p. 58) •  Three prisoners: A, B, and C. •  One will be executed next day, two released. •  Prisoner A asks the guard to tell him the name
of one of the other two who will be released. •  Guard says that B will be released. (Assume that
the guard is telling the truth.) •  A argues: before my chances to be executed
were 1/3, now they are 1/2 since I know it’s
either me or C. What’s wrong with his
reasoning? Prisoner’s Dilemma: A few remarks •  The prisoners only know that one of them will be
executed, but do not know which one. Thus, from
their point of view, a reasonable model is a discrete
uniform probability law that assigns each of them
probability 1/3 to be executed. •  The guard knows which one of them will be executed. •  Since A already knows that either B or C will be
released, it does not seem like knowing that B will be
released should inﬂuence A’s chances in any way. In
fact, A’s probability to be executed is sJll 1/3 (from A’s
point of view), as shown in the next few slides. Prisoner’s Dilemma: SoluJon •  The guard’s response needs to be included in the probabilisJc
model. •  Let Ei={prisoner i will be executed} for i=A,B,C •  Let Gj={guard names prisoner j} for j=B,C Prisoner’s Dilemma: SoluJon •  The guard’s response needs to be included in the probabilisJc
model. •  Let Ei={prisoner i will be executed} for i=A,B,C •  Let Gj={guard names prisoner j} for j=B,C 1/3 1/3 1/3 EA EB E C Prisoner’s Dilemma: SoluJon •  The guard’s response needs to be included in the probabilisJc
model. •  Let Ei={prisoner i will be executed} for i=A,B,C •  Let Gj={guard names prisoner j} for j=B,C 1/3 1/3 1/3 EA EB E C 1/2 1/2 EA∩GB
EA∩GC
Prisoner’s Dilemma: SoluJon •  The guard’s response needs to be included in the probabilisJc
model. •  Let Ei={prisoner i will be executed} for i=A,B,C •  Let Gj={guard names prisoner j} for j=B,C 1/3 1/3 1/3 EA 1/2 1/2 EB 0 1 E C EA∩GB
EA∩GC
EB∩GB
EB∩GC
Prisoner’s Dilemma: SoluJon •  The guard’s response needs to be included in the probabilisJc
model. •  Let Ei={prisoner i will be executed} for i=A,B,C •  Let Gj={guard names prisoner j} for j=B,C 1/3 1/3 1/3 EA 1/2 1/2 EB 0 1 E C 1 0 EA∩GB
EA∩GC
EB∩GB
EB∩GC
EC∩GB
EC∩GC
Prisoner’s Dilemma: SoluJon •  The guard’s response needs to be included in the probabilisJc
model. •  Let Ei={prisoner i will be executed} for i=A,B,C •  Let Gj={guard names prisoner j} for j=B,C 1/3 1/3 1/3 EA 1/2 1/2 EB 0 1 E C 1 0 EA∩GB
1/6 EA∩GC
EB∩GB
1/6 0 EB∩GC
1/3 EC∩GB
1/3 0 EC∩GC
Prisoner’s Dilemma: SoluJon •  The guard’s response needs to be included in the probabilisJc
model. •  Let Ei={prisoner i will be executed} for i=A,B,C •  Let Gj={guard names prisoner j} for j=B,C 1/3 1/3 1/3 EA 1/2 1/2 EB 0 1 E C 1 0 EA∩GB
1/6 EA∩GC
EB∩GB
1/6 0 EB∩GC
1/3 EC∩GB
1/3 0 EC∩GC
P(EA|GB) = P(EA∩GB)/P(GB) = (1/6)/(1/6 + 0 + 1/3) = 1/3
The Monty Hall Puzzle (Ex. 1.12) •  Prize behind one of three doors. •  Contestant picks a door. •  Host (who knows where the prize is) opens
one of the remaining two doors which does
not have the prize. •  Contestant is oﬀered an opportunity to stay
with his door, or to switch to another door. •  Stay or switch? The Monty Hall Puzzle: SoluJon •  If stay, P(win) = 1/3 The Monty Hall Puzzle: SoluJon •  If stay, P(win) = 1/3 •  If switch, the only way to lose is if iniJally
pointed to the door with prize: P(lose) = 1/3
and so P(win) = 2/3 The Monty Hall Puzzle: SoluJon •  If stay, P(win) = 1/3 •  If switch, the only way to lose is if iniJally
pointed to the door with prize: P(lose) = 1/3
and so P(win) = 2/3 •  Conclusion: must switch! •  Switching is advantageous because the host’s
acJon tells you something. If you iniJally
picked a door with no prize, he is forced to
open the other door with no prize. The Monty Hall Puzzle: Discussion •  Crucial parts of the problem statement: –  the host knows where the prize is –  he must open the door with no prize The Monty Hall Puzzle: Discussion •  Crucial parts of the problem statement: –  the host knows where the prize is –  he must open the door with no prize •  Thus, if you have chosen a door with no prize,
you are forcing him to open the only other
door with no prize and thus show you where
the prize is. The Monty Hall Puzzle: Another
SoluJon •  Use the total probability theorem to evaluate the probabiliJes of
winning under the two strategies. The Monty Hall Puzzle: Another
SoluJon •  Use the total probability theorem to evaluate the probabiliJes of
winning under the two strategies. •  Let W = “win” and N = “originally point to a door with no prize”. The Monty Hall Puzzle: Another
SoluJon •  Use the total probability theorem to evaluate the probabiliJes of
winning under the two strategies. •  Let W = “win” and N = “originally point to a door with no prize”. •  P(N) = 2/3; P(Nc) = 1/3. The Monty Hall Puzzle: Another
SoluJon •  Use the total probability theorem to evaluate the probabiliJes of
winning under the two strategies. •  Let W = “win” and N = “originally point to a door with no prize”. •  P(N) = 2/3; P(Nc) = 1/3. •  If you do not switch, P(W|N) = 0 and P(W|Nc) = 1. The Monty Hall Puzzle: Another
SoluJon •  Use the total probability theorem to evaluate the probabiliJes of
winning under the two strategies. •  Let W = “win” and N = “originally point to a door with no prize”. •  P(N) = 2/3; P(Nc) = 1/3. •  If you do not switch, P(W|N) = 0 and P(W|Nc) = 1. •  So, if you do not switch, P(W) = P(W|N)P(N) + P(W|Nc)P(Nc) = 1/3. The Monty Hall Puzzle: Another
SoluJon •  Use the total probability theorem to evaluate the probabiliJes of
winning under the two strategies. •  Let W = “win” and N = “originally point to a door with no prize”. •  P(N) = 2/3; P(Nc) = 1/3. •  If you do not switch, P(W|N) = 0 and P(W|Nc) = 1. •  So, if you do not switch, P(W) = P(W|N)P(N) + P(W|Nc)P(Nc) = 1/3. •  If you switch, P(W|N) = 1 because if you originally choose a door
with no prize the host is forced to open the only other door with
no prize. The Monty Hall Puzzle: Another
SoluJon •  Use the total probability theorem to evaluate the probabiliJes of
winning under the two strategies. •  Let W = “win” and N = “originally point to a door with no prize”. •  P(N) = 2/3; P(Nc) = 1/3. •  If you do not switch, P(W|N) = 0 and P(W|Nc) = 1. •  So, if you do not switch, P(W) = P(W|N)P(N) + P(W|Nc)P(Nc) = 1/3. •  If you switch, P(W|N) = 1 because if you originally choose a door
with no prize the host is forced to open the only other door with
no prize. •  If you switch, P(W|Nc) = 0 because if you originally choose the
door with the prize, you will switch out of it and lose. The Monty Hall Puzzle: Another
SoluJon •  Use the total probability theorem to evaluate the probabiliJes of
winning under the two strategies. •  Let W = “win” and N = “originally point to a door with no prize”. •  P(N) = 2/3; P(Nc) = 1/3. •  If you do not switch, P(W|N) = 0 and P(W|Nc) = 1. •  So, if you do not switch, P(W) = P(W|N)P(N) + P(W|Nc)P(Nc) = 1/3. •  If you switch, P(W|N) = 1 because if you originally choose a door
with no prize the host is forced to open the only other door with
no prize. •  If you switch, P(W|Nc) = 0 because if you originally choose the
door with the prize, you will switch out of it and lose. •  So, if you switch, P(W) = P(W|N)P(N) + P(W|Nc)P(Nc) = 2/3. Two‐Envelopes Puzzle (p. 58) •  You are handed two envelopes, each
containing an integer number of dollars,
unknown to you. •  The two amounts are diﬀerent. •  You select at random one envelope and look
inside. •  You can either sJck with this envelope or take
the other envelope. Your objecJve is to get
the larger amount. •  Does it maker what you do? Two‐Envelopes Puzzle: SoluJon •  Make independent ﬂips of a fair coin unJl heads come up for the
ﬁrst Jme. Let X = 1/2 + number of tosses to get ﬁrst H. •  Strategy: –  If the amount in your envelope > X, stay –  If the amount in your envelope < X, switch Two‐Envelopes Puzzle: SoluJon •  Make independent ﬂips of a fair coin unJl heads come up for the
ﬁrst Jme. Let X = 1/2 + number of tosses to get ﬁrst H. •  Strategy: –  If the amount in your envelope > X, stay –  If the amount in your envelope < X, switch •  Denote the two amounts d and D, d < D Two‐Envelopes Puzzle: SoluJon •  Make independent ﬂips of a fair coin unJl heads come up for the
ﬁrst Jme. Let X = 1/2 + number of tosses to get ﬁrst H. •  Strategy: –  If the amount in your envelope > X, stay –  If the amount in your envelope < X, switch •  Denote the two amounts d and D, d < D •  Case 1: X < d ‐‐‐ always stay ‐‐‐ win with probability 1/2 Two‐Envelopes Puzzle: SoluJon •  Make independent ﬂips of a fair coin unJl heads come up for the
ﬁrst Jme. Let X = 1/2 + number of tosses to get ﬁrst H. •  Strategy: –  If the amount in your envelope > X, stay –  If the amount in your envelope < X, switch •  Denote the two amounts d and D, d < D •  Case 1: X < d ‐‐‐ always stay ‐‐‐ win with probability 1/2 •  Case 2: X > D ‐‐‐ always switch ‐‐‐ win with probability 1/2 Two‐Envelopes Puzzle: SoluJon •  Make independent ﬂips of a fair coin unJl heads come up for the
ﬁrst Jme. Let X = 1/2 + number of tosses to get ﬁrst H. •  Strategy: –  If the amount in your envelope > X, stay –  If the amount in your envelope < X, switch • 
• 
• 
• 
Denote the two amounts d and D, d < D Case 1: X < d ‐‐‐ always stay ‐‐‐ win with probability 1/2 Case 2: X > D ‐‐‐ always switch ‐‐‐ win with probability 1/2 Case 3: d < X < D ‐‐‐ stay if pick D, switch if pick d ‐‐‐ win! Two‐Envelopes Puzzle: SoluJon •  Make independent ﬂips of a fair coin unJl heads come up for the
ﬁrst Jme. Let X = 1/2 + number of tosses to get ﬁrst H. •  Strategy: –  If the amount in your envelope > X, stay –  If the amount in your envelope < X, switch •  Denote the two amounts d and D, d < D •  Case 1: X < d ‐‐‐ always stay ‐‐‐ win with probability 1/2 •  Case 2: X > D ‐‐‐ always switch ‐‐‐ win with probability 1/2 •  Case 3: d < X < D ‐‐‐ stay if pick D, switch if pick d ‐‐‐ win! •  P(win) = P(win | X<d) P(X<d) + P(win | X>D) P(X>D) + P(win | d<X<D) P(d<X<D) (by total probability theorem) Two‐Envelopes Puzzle: SoluJon •  Make independent ﬂips of a fair coin unJl heads come up for the
ﬁrst Jme. Let X = 1/2 + number of tosses to get ﬁrst H. •  Strategy: –  If the amount in your envelope > X, stay –  If the amount in your envelope < X, switch •  Denote the two amounts d and D, d < D •  Case 1: X < d ‐‐‐ always stay ‐‐‐ win with probability 1/2 •  Case 2: X > D ‐‐‐ always switch ‐‐‐ win with probability 1/2 •  Case 3: d < X < D ‐‐‐ stay if pick D, switch if pick d ‐‐‐ win! •  P(win) = P(win | X<d) P(X<d) + P(win | X>D) P(X>D) + P(win | d<X<D) P(d<X<D) = 1/2 P(X<d) + 1/2 P(X>D) + P(d<X<D) Two‐Envelopes Puzzle: SoluJon •  Make independent ﬂips of a fair coin unJl heads come up for the
ﬁrst Jme. Let X = 1/2 + number of tosses to get ﬁrst H. •  Strategy: –  If the amount in your envelope > X, stay –  If the amount in your envelope < X, switch •  Denote the two amounts d and D, d < D •  Case 1: X < d ‐‐‐ always stay ‐‐‐ win with probability 1/2 •  Case 2: X > D ‐‐‐ always switch ‐‐‐ win with probability 1/2 •  Case 3: d < X < D ‐‐‐ stay if pick D, switch if pick d ‐‐‐ win! •  P(win) = P(win | X<d) P(X<d) + P(win | X>D) P(X>D) + P(win | d<X<D) P(d<X<D) = 1/2 P(X<d) + 1/2 P(X>D) + P(d<X<D) = 1/2 (P(X<d) + P(X>D) + P(d<X<D)) + 1/2 P(d<X<D) =1 Two‐Envelopes Puzzle: SoluJon •  Make independent ﬂips of a fair coin unJl heads come up for the
ﬁrst Jme. Let X = 1/2 + number of tosses to get ﬁrst H. •  Strategy: –  If the amount in your envelope > X, stay –  If the amount in your envelope < X, switch •  Denote the two amounts d and D, d < D •  Case 1: X < d ‐‐‐ always stay ‐‐‐ win with probability 1/2 •  Case 2: X > D ‐‐‐ always switch ‐‐‐ win with probability 1/2 •  Case 3: d < X < D ‐‐‐ stay if pick D, switch if pick d ‐‐‐ win! •  P(win) = P(win | X<d) P(X<d) + P(win | X>D) P(X>D) + P(win | d<X<D) P(d<X<D) = 1/2 P(X<d) + 1/2 P(X>D) + P(d<X<D) = 1/2 (P(X<d) + P(X>D) + P(d<X<D)) + 1/2 P(d<X<D) = 1/2 (1 + P(d<X<D)) > 1/2 Two‐Envelopes Puzzle: Comment •  Note that X can be any random variable having
non‐zero values at 3/2, 5/2, 7/2 (or, in fact, at
any point(s) on ]1,2[, at any point(s) on ]2,3[,
etc.) 