PSTAT 5A: Introduction to Probability

Lecture 5 - Conditional Probability, Independence & Bayes Theorem

Narjes Mathlouthi

2025-07-10

Today’s Learning Objectives

By the end of this lecture, you will be able to:

• Define probability and understand its basic properties
• Identify sample spaces and events
• Apply fundamental probability rules
• Calculate conditional probabilities

Practice Problem 1

A standard deck has 52 cards. What is the probability of drawing:

1. A heart \(\heartsuit\)?
2. A face card (Jack, Queen, King)?
3. The ace of spades \(\spadesuit\)?

Solution.

1. \(P(\text{heart}) = \frac{13}{52} = \frac{1}{4}\)

2. \(P(\text{face card}) = \frac{12}{52} = \frac{3}{13}\)

3. \(P(\text{ace of spades}) = \frac{1}{52}\)

The Addition Rule

For any two events \(A\) and \(B\):

\[P(A \cup B) = P(A) + P(B) - P(A \cap B)\]

Why subtract \(P(A \cap B)\)?

Solution. We don’t want to double-count outcomes that are in both events

Addition Rule Example

Drawing from a standard deck:

• \(A\): Drawing a heart \(\heartsuit\) (\(P(A) = \frac{13}{52}\))

• \(B\): Drawing a face card (\(P(B) = \frac{12}{52}\))

• What’s \(P(A \cup B)\) (heart OR face card)?

Solution. \(P(A \cap B) = \frac{3}{52}\) (face cards that are hearts)

\(P(A \cup B) = \frac{13}{52} + \frac{12}{52} - \frac{3}{52} = \frac{22}{52} = \frac{11}{26}\)

Conditional Probability

🎯Conditional probability is the probability of event \(A\) given that event \(B\) has occurred

\[P(A|B) = \frac{P(A \cap B)}{P(B)}\]

provided \(P(B) > 0\)

Conditional Probability Interpretation

\(P(A|B)\) means:

• We know event \(B\) has occurred

• What’s the probability that \(A\) also occurred?

• We “restrict” our sample space to only outcomes in \(B\)

Conditional Probability Example

Drawing a card from a standard deck:

• \(A\): Card is a heart \(\heartsuit\)

• \(B\): Card is red

• Q: Find \(P(A|B)\)

Solution. \(P(A \cap B) = P(\text{heart}) = \frac{13}{52}\)

\(P(B) = P(\text{red}) = \frac{26}{52}\)

\(P(A|B) = \frac{13/52}{26/52} = \frac{13}{26} = \frac{1}{2}\)

Independence

🎯 Definition Events \(A\) and \(B\) are independent if:

\[P(A|B) = P(A)\]

or equivalently:

\[P(A \cap B) = P(A) \times P(B)\]

Knowing that \(B\) occurred doesn’t change the probability of \(A\)

Independence Example

Two coin flips:

• \(A\): First flip is heads

• \(B\): Second flip is heads

Q: Are \(A\) and \(B\) independent?

Solution. \(P(A) = \frac{1}{2}\), \(P(B) = \frac{1}{2}\)

\(P(A \cap B) = P(\text{HH}) = \frac{1}{4}\)

\(P(A) \times P(B) = \frac{1}{2} \times \frac{1}{2} = \frac{1}{4}\)

Yes, they are independent!

Mutually Exclusive vs. Independent

• Mutually Exclusive (left): the circles A and B do not overlap, so \(P(A\cap B)=0\).

• Independent (right): the circles overlap, and we’ve sized the intersection so that \(P(A\cap B)=P(A)\,P(B)\).

Mutually Exclusive vs. Independent Example

Draw a single card from a 52-card deck:

• Let A={“draw an Ace”}, so P(A)=4/52.

• Let B={“draw a King”}, so P(B)=4/52.

Q: What is \(P(A\cap B)\) ?

Solution. They’re disjoint (you can’t draw an Ace and a King), so \(P(A\cap B) = 0\).

But \(P(A)\,P(B) = \frac{4}{52}\times\frac{4}{52} = \frac{16}{2704} \neq 0\).

Hence, \(P(A\cap B)\neq P(A)P(B)\), so they’re not independent.

Multiplication Rule

General case: \(P(A \cap B) = P(A) \times P(B|A)\)

Independent events: \(P(A \cap B) = P(A) \times P(B)\)

Tree Diagrams

🎯 Definition Tree diagrams help visualize sequential events and calculate probabilities.

Tree Diagram Examples

Practice Problem 2

A jar contains 5 red balls and 3 blue balls. Two balls are drawn without replacement.

1. What’s the probability both balls are red?

2. What’s the probability the first is red and second is blue?

Solution.

1. \(P(\text{both red}) = \frac{5}{8} \times \frac{4}{7} = \frac{20}{56} = \frac{5}{14}\)

2. \(P(\text{red then blue}) = \frac{5}{8} \times \frac{3}{7} = \frac{15}{56}\)

Law of Total Probability

🎯 Definition

If events \(B_1, B_2, \ldots, B_n\) form a partition of the sample space, then:

\[P(A) = P(A|B_1)P(B_1) + P(A|B_2)P(B_2) + \cdots + P(A|B_n)P(B_n)\]

Law of Total Probability Example

A factory has two machines:

• Machine 1: Produces 60% of items, 5% defective

• Machine 2: Produces 40% of items, 3% defective

Q: What’s the overall probability an item is defective?

Solution. \(P(\text{defective}) = P(D|M_1)P(M_1) + P(D|M_2)P(M_2)\)

\(= 0.05 \times 0.6 + 0.03 \times 0.4 = 0.03 + 0.012 = 0.042\)

Bayes’ Theorem

🎯 Definition \[P(A|B) = \frac{P(B|A) \times P(A)}{P(B)}\]

This allows us to “reverse” conditional probabilities

Named after Thomas Bayes (1701-1761)

Bayes’ Theorem Components

• \(A,B\): Events
• \(P(A|B)\): Posterior probability - what we want to find
• \(P(B|A)\): Likelihood - given \(A\), probability of observing \(B\)
• \(P(A)\): Prior probability - initial probability of \(A\)
• \(P(B)\): Marginal probability - total probability of \(B\)

Bayes’ Theorem Example

Medical test for a disease: ¹

• Disease affects 1% of population

• Test is 95% accurate for sick people

• Test is 90% accurate for healthy people

Q:If someone tests positive, what’s the probability they have the disease?

1. \(P(\neg D \,\wedge\, \text{Negative}) \;=\;P(\neg D)\times P(\text{Negative}\mid \neg D) \;=\;0.99\times0.90 \;=\;0.891\;(89.1\%)\)

Bayes’ Theorem Solution

Let:

• \(D\): Person has disease

• \(T^+\): Test is positive

Given:

• \(P(D) = 0.01\)

• \(P(T^+|D) = 0.95\)

• \(P(T^-|D^c) = 0.90\), so \(P(T^+|D^c) = 0.10\)

Solution. \(P(T^+) = P(T^+|D)P(D) + P(T^+|D^c)P(D^c)\)

\(= 0.95 \times 0.01 + 0.10 \times 0.99 = 0.1085\)

Bayes’ Theorem Solution (cont.)

\[P(D|T^+) = \frac{P(T^+|D) \times P(D)}{P(T^+)} = \frac{0.95 \times 0.01}{0.1085} \approx 0.088\]

Surprising result: Even with a positive test, there’s only an 8.8% chance of having the disease!

This is due to the low base rate of the disease

Common Probability Mistakes

• Confusing \(P(A|B)\) with \(P(B|A)\)

Prosecutor’s fallacy is a specific error in interpreting conditional probabilities. Confusing

\(P(\text{Evidence}\mid\text{Innocent}) \quad\text{with}\quad P(\text{Innocent}\mid\text{Evidence})\).

Ex: OJ Simpson Case ¹

1. the DNA evidence in the O. J. Simpson trial are a classic example of the prosecutor’s fallacy. Prosecutors highlighted that the chance of a random person matching the crime-scene DNA was “one in 170 million,” then implied (or let the jury infer) that Simpson therefore had a 1 in 170 million chance of being innocent.

Common Probability Mistakes

• Assuming independence when events are dependent

• Ignoring base rates (as in the medical test example)

Base rate fallacy is when you ignore or underweight the prior probability \(P(H)\) of a hypothesis, focusing only on the new evidence \(E\).

• Double counting in union calculations

Practice Problem 3

Two fair dice are rolled. Find:

1. \(P(\text{sum} = 7)\)
2. \(P(\text{sum} = 7 | \text{first die shows 3})\)
3. Are these events independent?

Solution.

1. 6 ways out of 36: \(P(\text{sum} = 7) = \frac{6}{36} = \frac{1}{6}\)

2. Given first die is 3, need second die to be 4: \(P(\text{sum} = 7 | \text{first} = 3) = \frac{1}{6}\)

3. Yes, they’re independent since \(P(A|B) = P(A)\)

Real-World Applications

Medical Diagnosis: Using Bayes’ theorem for test interpretation

Quality Control: Probability of defective items

Finance: Risk assessment and portfolio theory

Sports: Probability of wins, fantasy sports

Insurance: Calculating premiums based on risk

Key Formulas Summary

• Basic probability: \(P(A) = \frac{\text{favorable outcomes}}{\text{total outcomes}}\)
• Complement: \(P(A^c) = 1 - P(A)\)
• Addition: \(P(A \cup B) = P(A) + P(B) - P(A \cap B)\)
• Conditional: \(P(A|B) = \frac{P(A \cap B)}{P(B)}\)
• Independence: \(P(A \cap B) = P(A) \times P(B)\)
• Bayes’: \(P(A|B) = \frac{P(B|A) \times P(A)}{P(B)}\)

Problem-Solving Strategy

1. Identify the sample space and events
2. Determine if events are independent or mutually exclusive
3. Choose the appropriate rule or formula
4. Calculate step by step
5. Check if your answer makes sense

Common Questions

Q1.: “Why isn’t \(P(A \cup B) = P(A) + P(B)\) always?”

A: We’d double-count outcomes in both events

Q2.: “How do I know if events are independent?”

A: Check if \(P(A|B) = P(A)\) or if \(P(A \cap B) = P(A) \times P(B)\)

Q3.: “When do I use Bayes’ theorem?”

A: When you want to “reverse” a conditional probability

Q3 note (Bayes Example)

• Forward: I know my test picks up disease 95% of the time ⇒ \(P(+\mid D)=0.95\).

• Reverse: I want the chance I really have the disease when the test is positive ⇒ \(P(D\mid +)\).

Looking Ahead

Next lecture:

• Counting

• Random Variables and Probability Distributions

• Discrete vs. continuous random variables

• Expected value and variance

Final Thoughts

Probability is the foundation of statistics:

• Helps us quantify uncertainty

• Provides tools for making decisions with incomplete information

• Essential for understanding statistical inference

Practice: The key to mastering probability is working through many problems!

Questions?

Office Hours: Thursday’s 11 AM On Zoom (Link on Canvas)

Email: nmathlouthi@ucsb.edu

Next Class: Conditional Probability & Bayes Theorem

Resources

• Read Chapter 3 in course textbook

• Elements of Set Theory for Probability

• Probability Models and Axioms

• Interactive Set Theory & Conditional Probability