Math (P)refresher Day 5: Great Expectations, Part I September 2000 Today's Topics : Probability Conditional Probability and Independence Bayes' Rule 1 Probability Probability: Many events or outcomes are random. In every day speech, we say that we are about the outcome of random events. Probability is a formal model of uncertainty which provides a measure of uncertainty governed by a particular set of rules. A dierent model of uncertainty would, of course, have a dierent set of rules and measures. Our focus on probability is justied because it has proven to be a particularly useful model of uncertainty. Sample Space: A set or collection of all possible outcomes. Outcomes in the set can be discrete elements or points along a continuous interval. Examples: 1. Discrete: the numbers on a die, the number of possible wars that could occur each year, whether a vote cast is republican or democrat. 2. Continuous: GNP, arms spending, age. Probability Distribution: A probability distribution on a sample space S is a specication of numbers Pr(A) that satisfy Axioms 1{3. Axioms of Probability: It's necessary to assign a number Pr(A) to each event A in the sample space S . 1. Axiom: For any event A, Pr(A) 0. 2. Axiom: Pr(S ) = 1 3. Axiom: 1 For any1innite sequence of disjoint events A1 ; A2 ; : : : , S P Pr Ai = Pr(Ai ) uncertain i=1 i=1 Basic Theorems of Probability: 1. Pr(;) = 0 2. Forany sequence of n disjoint events A ; A ; : : : ; An , n P n S Pr Ai = Pr(Ai ) i i 3. Pr(A) = 1 ; Pr(A) 1 =1 2 =1 Much of the material and examples for this lecture are taken from Wackerly, Mendenhall, & Scheaer (1996) Mathematical Statistics with Applications, Degroot (1985) Probability and Statistics, Morrow (1994) Game Theory for Political Scientists, King (1989) Unifying Political Methodology, and Ross (1987) Introduction to Probability and Statistics for Scientists and Engineers. 2 Math (P)refresher: Great Expectations 4. For any event A, 0 Pr(A) 1. 5. If A B , then Pr(A) Pr(B ). 6. For any two events A and B , Pr(A [ B ) = Pr(A) + Pr(B ) ; Pr(A \ B ) Examples: Let's assume we have an evenly-balanced, six-sided die. Then, 1. Sample space S = f1; 2; 3; 4; 5; 6g 2. Pr(1) = = Pr(6) = 1=6 3. Pr(;) = Pr(7) = 0 4. Pr (f1; 3; 5g) = 1=6 + 1=6 + 1=6 = 1=2 5. Pr f1; 2g = Pr (f3; 4; 5; 6g) = 2=3 6. Let B = S and A = f1; 2; 3; 4; 5g B . Then Pr(A) = 5=6 < Pr(B ) = 1. 7. Let A = f1; 2; 3g and B = f2; 4; 6g. Then A [ B = f1; 2; 3; 4; 6g, A \ B = f2g, and Pr(A [ B ) = Pr(A) + Pr(B ) ; Pr(A \ B ) = 3=6 + 3=6 ; 1=6 = 5=6 2 Conditional Probability and Independence Conditional Probability: The conditional probability Pr(AjB ) of an event A is the probability of A, given that another event B has occurred. It is calculated as A \ B) Pr(AjB ) = Pr(Pr( B) Example: Assume A and B occur with the following frequencies: and let nab + nab + nab + nab = N . Then 1. 2. 3. 4. 5. A A B nab nab B nab nab Pr(A) nabN+nab Pr(B ) nabN+nab Pr(A \ B ) nNab Pr(AjB ) Pr(Pr(AB\B) ) = nabn+abnab Pr(B jA) Pr(Pr(AA\B) ) = nabn+abnab Example: A six-sided die is rolled. What is the probability of a 1, given the outcome is an odd number? Let A = f1g, B = f1; 3; 5g, and A \ B = f1g. Then, Pr(AjB ) = AB\B = = = 1=3. = Multiplicative Law of Probability: The probability of the intersection of two events A and B is Pr(A \ B ) = Pr(A) Pr(B jA) = Pr(B ) Pr(AjB ) Pr( Pr( 1 6 1 2 which follows directly from the denition of conditional probability. ) ) 3 Math (P)refresher: Great Expectations Independence: If the occurrence or nonoccurrence of either events A and B have no eect on the occurrence or nonoccurrence of the other, then A and B are independent. If A and B are independent, then 1. Pr(AjB ) = Pr(A) 2. Pr(B jA) = Pr(B ) 3. Pr(A \ B ) = Pr(A) Pr(B ) Calculating the Probability of an Event Using the Event-Composition Method: The event-composition method for calculating the probability of an event A involves expressing A as a composition involving the unions and/or intersections of other events. Then use the laws of probability to to nd Pr(A). The steps used in the event-composition method are: 1. Dene the experiment. 2. Identify the general nature of the sample points. 3. Write an equation expressing the event of interest A as a composition of two or more events, using unions, intersections, and/or complements. 4. Apply the additive and multiplicative laws of probability to the compositions obtained in step 3 to nd Pr(A). Examples: 1. 2. 3. 4. 3 Bayes' Rule Law of Total Probability: Let S be the sample space of some experiment and let the disjoint k events B ; : : : ; Bk partition S . If A is some other event in S , then the events AB ; AB ; : : : ; ABk will form a partition of A and we can write A as A = (AB ) [ [ (ABk ) Since the k events are disjoint, 1 1 2 1 Pr(A) = = k X i=1 k X i=1 Pr(ABi ) Pr(Bi ) Pr(AjBi ) Sometimes it is easier to calculate the conditional probabilities and sum them than it is to calculate Pr(A) directly. 4 Math (P)refresher: Great Expectations Bayes Rule: Assume that events B ; : : : ; Bk form a partition of the space S . Then ABj ) = Pr(Bj ) Pr(AjBj ) Pr(Bj jA) = Pr( k Pr(A) P Pr(Bi ) Pr(AjBi ) 1 i=1 If there are only two states of B , then this is just B1 ) Pr(AjB1 ) Pr(B1 jA) = Pr(B ) Pr(Pr( AjB1 ) + Pr(B2 ) Pr(AjB2 ) 1 Bayes rule determines the posterior probability of a state or type Pr(Bj jA) by calculating the probability Pr(ABj ) that both the event A and the state Bj will occur and dividing it by the probability that the event will occur regardless of the state (by summing across all Bi ). Often Bayes' rule is used when one wants to calculate a posterior probability about the \state" or type of an object, given that some event has occurred. The states could be something like Normal/Defective, Normal/Diseased, Democrat/Republican, etc. The event on which one conditions could be something like a sampling from a batch of components, a test for a disease, or a question about a policy position. Prior and Posterior Probabilities: In the above, Pr(B1 ) is often called the prior probability, since it's the probability of B1 before anything else is known. Pr(B1 jA) is called the posterior probability, since it's the probability after other information is taken into account. Examples: 1. A test for cancer correctly detects it 90% of the time, but incorrectly identies a person as having cancer 10% of the time. If 10% of all people have cancer at any given time, what is the probability that a person who tests positive actually has cancer? Let C = fCancerg, C = fNo cancerg, + = fTests positiveg, and ; = fTests negativeg. Then Pr(C ) Pr(+jC ) :9) Pr(C j+) = = (:1)(:(9):1)( + (:9)(:1) = 0:5 Pr(C ) Pr(+jC ) + Pr(C ) Pr(+jC ) 2. In Boston, 30% of the people are conservatives, 50% are liberals, and 20% are independents. In the last election, 65% of conservatives, 82% of liberals, and 50% of independents voted. If a person in Boston is selected at random and we learn that s/he did not vote last election, what is the probability s/he is a liberal? Let C , L, and I represent conservatives, liberals, and independents, respectively. And, let V = fVotedg and V = fDidn't voteg. Then, Pr(L) Pr(V jL) Pr(C ) Pr(V jC ) + Pr(L) Pr(V jL) + Pr(I ) Pr(V jI ) :18) = (:3)(:35) +((::5)( 5)(:18) + (:2)(:5) = 18 59 Pr(LjV ) =