Chapter 2: CONDITIONAL PROBABILITY

Charles Boncelet, “Probability, Statistics, and Random Signals,"Oxford University Press, 2016. ISBN: 978-0-19-020051-0Chapter 2: CONDITIONAL PROBABILITYSections2.1 Definitions of Conditional Probability2.2 Law of Total Probability and Bayes Theorem2.3 Example: Urn Models2.4 Example: A Binary Channel2.5 Example: Drug Testing2.6 Example: A Diamond NetworkSummaryProblemsNotes and figures are based on or taken from materials in the course textbook: Charles Boncelet,Probability, Statistics, and Random Signals, Oxford University Press, February 2016.B.J. Bazuin, Fall 20201 of 25ECE 3800

Joint Probability – Compound ExperimentsDefining probability based on multiple events two classes for considerations. Independent experiments: The outcome of one experiment is not affected by pastor future experiments.o flipping coinso repeating an experiment after initial conditions have been restoredo Note: these problems are typically easier to solve Dependent experiments: The result of each subsequent experiment is affected bythe results of previous experiments.o drawing cards from a deck of cardso drawing strawso selecting names from a hato for each subsequent experiment, the previous results change the possibleoutcomes for the next event.o Note: these problems can be very difficult to solve (the “next experiment”changes based on previous outcomes!)Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,Probability, Statistics, and Random Signals, Oxford University Press, February 2016.B.J. Bazuin, Fall 20202 of 25ECE 3800

Conditional ProbabilityDefining the conditional probability of event A given that event B has occurred.Using a Venn diagram, we know that B has occurred then the probability that A hasoccurred given B must relate to the area of the intersection of A and B Pr A B Pr A B Pr B , for Pr B 0ThereforePr A B Pr A B , for Pr B 0Pr B For elementary events,Pr A B Pr A B Pr A, B , for Pr B 0Pr B Pr B Special cases for A B , B A , and B A .Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,Probability, Statistics, and Random Signals, Oxford University Press, February 2016.B.J. Bazuin, Fall 20203 of 25ECE 3800

Special cases for A B , B A , and B A . If A is a subset of B, then the conditional probability must bePr A B Pr A B Pr A , for A BPr B Pr B Therefore, it can be said thatPr A B Pr A B Pr A Pr A , for A BPr B Pr B If B is a subset of A, then the conditional probability becomesPr A B Pr A B Pr B 1 , for B APr B Pr B If A and B are mutually exclusive,Pr A B Pr A B 0 0 , for B A Pr B Pr B Conditional probabilities are generally not symmetric!Pr A B Pr A B Pr B , for Pr B 0Pr A B Pr B A Pr A , for Pr A 0andPr A B , for Pr B 0Pr B Pr A B Pr B A , for Pr A 0Pr A Pr A B Pr B A Pr A B Pr B Pr A B Pr B Pr B Pr A B Pr A Pr B Pr B Pr A Pr A Therefore, we would expect (unless Pr(A) and Pr(B) are equal)Pr B A Pr A B Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,Probability, Statistics, and Random Signals, Oxford University Press, February 2016.B.J. Bazuin, Fall 20204 of 25ECE 3800

Total ProbabilityFor a space, S, that consists of multiple mutually exclusive events, the probability of arandom event, B, occurring in space S, can be described based on the conditionalprobabilities associated with each of the possible events.Proof:S A1 A2 A3 AnandAi A j , for i jB B S B A1 A2 A3 An B A1 B A2 B A3 B An Pr B Pr B A1 Pr B A2 Pr B A3 Pr B An ButPr B Ai Pr B Ai Pr Ai , for Pr Ai 0ThereforePr B Pr B A1 Pr A1 Pr B A2 Pr A2 Pr B An Pr An Remember your math properties: distributive, associative, commutative etc. applied to settheory.Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,Probability, Statistics, and Random Signals, Oxford University Press, February 2016.B.J. Bazuin, Fall 20205 of 25ECE 3800

Continuing ConceptsConditional Probability When the probability of an event depends upon prior events.If trials are performed without replacement and/or the initial conditions arenot restored, you expect trial outcomes to be dependent on prior results orconditions.Pr A B Pr A when A follows B The joint probability is.Pr A, B Pr B, A Pr A B Pr B Pr B A Pr A Applicable for objects that have multiple attributes and/or for trialsperformed without replacement.Experiment 3: A bag of marbles, draw 2 without replacement Experiment: Draw two marbles, without replacement Sample Space: {BB, BR, BY, RR, RB, RY, YB, YR}Therefore1st-rows \2nd-col1st-Blue1st-Red1st-YellowTotal 2ndMarble2nd-Blue2nd-Red2nd-Yellow 3 2 6 6 5 30 2 3 6 6 5 30 1 3 3 6 5 3036 3 2 6 6 5 30 2 1 2 6 5 30 1 2 2 6 5 3026 3 1 3 6 5 30 2 1 2 6 5 30 1 0 0 6 5 3016Total1stMarble36261666Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,Probability, Statistics, and Random Signals, Oxford University Press, February 2016.B.J. Bazuin, Fall 20206 of 25ECE 3800

Resistor Example: Joint and Conditional ProbabilityAssume we have a bunch of resistors (150) of various impedances and powers Similar to old textbook problems (more realistic resistor values)50 ohms100 ohms200 ohmsWatt Subtotal¼ watt40201070½ watt30205551 watt1010525ohm Subtotal805020150Each object has two attributes: impedance (ohms) and power rating (watts)Better Be Right Or Your Great Big Plan Goes Wrong. (p purple for violet)Bat Brained Resistor Order You Gotta Be Very Good WithMarginal Probabilities: (uses subtotals)Pr(¼ watt) 70/150Pr(½ watt) 55/150Pr(1 watt) 25/150Pr(50 ohms) 80/150Pr(100 ohms) 50/150Pr(200 ohms) 20/150These are called the marginal probabilities when fewer than all the attributes areconsidered (or don’t matter).Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,Probability, Statistics, and Random Signals, Oxford University Press, February 2016.B.J. Bazuin, Fall 20207 of 25ECE 3800

Joint Probabilities: divided each member of the table by 150!50 ohms100 ohms200 ohmsSubtotal¼ watt40/150 0.26620/150 0.13310/150 0.06670/150 0.466½ watt30/150 0.2020/150 0.1335/150 0.03355/150 0.3661 watt10/150 0.06610/150 0.0665/150 0.03325/150 0.166Subtotal80/150 0.53350/150 0.33320/150 0.133150/150 1.0These are called the joint probabilities when all unique attributes must be considered.(Concept of total probability things that sum to 1.0)Conditional Probabilities:When one attributes probability is determined based on the existence (or non-existence)of another attribute. Therefore,The probability of a ¼ watt resistor given that the impedance is 50 ohm.Pr(¼ watt given that the impedance is 50 ohms) Pr(¼ watt 50 ohms) 40/80 0.5050 ohms¼ watt40/80 0.50½ watt30/80 0.3751 watt10/80 0.125Total80/80 1.0Simple math that does not work to find the solution: (they are not independent)Pr(¼ watt) 70/150 andPr(50 ohms) 80/150Pr(¼ watt) x Pr(50 ohms) 70/150 x 80/150 56/225 0.249 NO!!! Not independent!!Math that does workPr A B 40Pr A B Pr A, B 150 40 0.508080Pr B Pr B 150Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,Probability, Statistics, and Random Signals, Oxford University Press, February 2016.B.J. Bazuin, Fall 20208 of 25ECE 3800

What about Pr(50 ohms given the power is ¼ watt)¼ watt50 ohms100 ohms200 ohmsTotal40/70 0.57120/70 0.28610/70 0.14370/70 1.0Pr(50 ohms ¼ watt) Pr(50 ¼) 40/70 0.571Pr A B 40Pr A B Pr A, B 150 40 0.5717070Pr B Pr B 150Can you determine?Pr(100, ½) Pr(100) Pr(50, ½) Pr(½ 50) Pr(50 ½) Pr( 1 ) Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,Probability, Statistics, and Random Signals, Oxford University Press, February 2016.B.J. Bazuin, Fall 20209 of 25ECE 3800

Using the “table” it is rather straight forward 50 ohms100 ohms¼ watt4020½ watt30201 watt1010Subtotal8050200 ohms105520Subtotal705525150Joint Probabilities Pr A B Pr A, B Pr(100, ½) Conditional Probabilities Pr A B Pr(½ 100) Pr(50, ½) Pr A B Pr A, B Pr B Pr B Pr(200 ½) Marginal Probability Pr B Pr B A1 Pr A1 Pr B An Pr An Pr( 1 ) Pr(100) Are there multiple ways to conceptually define such problems? Yes Relative Frequency Approach (statistics) Set Theory Approach (formal math) Venn Diagrams (pictures based on set theory)All ways to derive equations that form desired probabilities . The Relative Frequency Approach is the slowest and requires the most work!Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,Probability, Statistics, and Random Signals, Oxford University Press, February 2016.B.J. Bazuin, Fall 202010 of 25ECE 3800

A Priori and A Posteriori Probability (Sec. 2.2 Bayes Theorem)The probabilities defined for the expected outcomes, Pr Ai , are referred to as a prioriprobabilities (before the event). They describe the probability before the actualexperiment or experimental results are known.After an event has occurred, the outcome B is known. The probability of the eventbelonging to one of the expected outcomes can be defined asPr Ai B or from beforePr Ai B Pr B Ai Pr Ai Pr Ai B Pr B Pr Ai B Pr B Ai Pr Ai , for Pr B 0Pr B Using the concept of total probabilityPr B Pr B A1 Pr A1 Pr B A2 Pr A2 Pr B An Pr An We also have the following formsPr Ai B Pr B Ai Pr Ai Pr B A1 Pr A1 Pr B A2 Pr A2 Pr B An Pr An orPr A j B Pr B A j Pr A j n Pr B A Pr A i 11 Pr B A j Pr A j Pr B 1This probability is referred to as the a posteriori probability (after the event).It is also referred to as Bayes Theorem.Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,Probability, Statistics, and Random Signals, Oxford University Press, February 2016.B.J. Bazuin, Fall 202011 of 25ECE 3800

ExampleMore ResistorsBin 1Bin 2Bin 3Bin 4Bin 5Bin 6Subtotal10 ohm5000200800120010003700100 ohm300400600200800023001000 0020008600What is the probability of selecting a 10 ohm resistor from a random bin?Given Bin marginal probability Pr Bin # 16Pr 10 Bin1 5001000Pr 10 Bin 2 01000Pr 10 Bin3 2001000Pr 10 Bin 4 8001600Pr 10 Bin5 12002000Pr 10 Bin6 10002000Pr B Pr B A1 Pr A1 Pr B A2 Pr A2 Pr B An Pr An Pr B Pr B 500 10 1 200 1 800 1 1200 1 1000 1 1000 6 1000 6 1000 6 1600 6 2000 6 2000 65 1 0 1 2 1 5 1 6 1 5 1 23 1 0.383310 6 10 6 10 6 10 6 10 6 10 6 10 6Assuming a 10 ohm resistor is selected, what is the probability it came from bin 3?Pr Ai B Pr B Ai Pr Ai Pr B A1 Pr A1 Pr B A2 Pr A2 Pr B An Pr An Pr Bin3 10 Pr 10 Bin3 Pr Bin3 Pr 10 Bin1 Pr Bin1 Pr 10 Bin6 Pr Bin6 2 1Pr Bin3 10 10 6 0.086960.3833Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,Probability, Statistics, and Random Signals, Oxford University Press, February 2016.B.J. Bazuin, Fall 202012 of 25ECE 3800

Digital TransmissionsA digital communication system sends a sequence of 0 and 1, each of which are receivedat the other end of a link. Assume that the probability that 0 is received correctly is 0.90and that a 1 is received correctly is 0.90. Alternately, the probability that a 0 or 1 is notreceived correctly is 0.10 (the cross-over probability, ). Within the sequence, theprobability that a 0 is sent is 60% and that a one is sent is 40%. [S is Send and R isReceive]Pr S1 0.40Pr S 0 0.60Pr R1 S1 0.90 1 Pr R0 S 0 0.90 1 Pr R1 S 0 0.10 Pr R0 S1 0.10 Figure 1.7-1a) What is the probability that a zero is received?Total Probability: Pr B Pr B A1 Pr A1 Pr B A2 Pr A2 Pr B An Pr An Pr R0 Pr R0 S 0 Pr S 0 Pr R0 S1 Pr S1 Pr R0 0.90 0.60 0.10 0.40Pr R0 0.54 0.04 0.58b) What is the probability that a one is received?Pr R1 Pr R1 S 0 Pr S 0 Pr R1 S1 Pr S1 Pr R1 0.10 0.60 0.90 0.40Pr R1 0.06 0.36 0.42Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,Probability, Statistics, and Random Signals, Oxford University Press, February 2016.B.J. Bazuin, Fall 202013 of 25ECE 3800