Conditional Probability and Bayes' Theorem

Conditional probability answers the question "how likely is \( A \), given that we already know \( B \) has happened?" Bayes' theorem goes one step further and lets us reverse the condition, which is how doctors interpret test results and how spam filters decide what to block. Both ideas follow from one simple formula.

What Is Conditional Probability?

The probability of \( A \) given \( B \) is written \( P(A \mid B) \) and defined as

$$ P(A \mid B) = \frac{P(A \cap B)}{P(B)}, \qquad P(B) > 0 $$

The intuition: once we know \( B \) has happened, \( B \) becomes our new sample space. We then ask what fraction of \( B \) also belongs to \( A \).

Quick Example

Roll a fair die. Let \( A \) = "a 2" and \( B \) = "an even number". Without information, \( P(A) = \frac{1}{6} \). But given the result is even, only \( \{2, 4, 6\} \) are possible, so

$$ P(A \mid B) = \frac{1/6}{3/6} = \frac{1}{3} $$

The General Multiplication Rule

Rearranging the definition gives

$$ P(A \cap B) = P(B)\,P(A \mid B) = P(A)\,P(B \mid A) $$

This is the rule to use for dependent events, such as drawing without replacement. If \( A \) and \( B \) are independent, then \( P(A \mid B) = P(A) \), and the rule reduces to \( P(A \cap B) = P(A)P(B) \).

Worked Example 1: Drawing Without Replacement

Question: A bag contains 5 red and 3 blue balls. Two balls are drawn one after the other without replacement. Find (a) the probability that both are red, (b) the probability that the second ball is red.

Step 1: First draw. There are 8 balls, 5 red, so \( P(R_1) = \frac{5}{8} \).

Step 2: Second draw given the first was red. Now 7 balls remain, 4 of them red:

$$ P(R_2 \mid R_1) = \frac{4}{7} $$

Step 3: Part (a). Use the multiplication rule:

$$ P(R_1 \cap R_2) = \frac{5}{8}\times\frac{4}{7} = \frac{20}{56} = \frac{5}{14} $$

Step 4: Part (b). The second ball can be red in two ways: red then red, or blue then red.

$$ P(B_1 \cap R_2) = \frac{3}{8}\times\frac{5}{7} = \frac{15}{56} $$

$$ P(R_2) = \frac{20}{56} + \frac{15}{56} = \frac{35}{56} = \frac{5}{8} $$

Interesting result: without knowing the first ball, the second ball is red with the same probability as the first, \( \frac{5}{8} \). Step 4 is an example of the next idea.

The Law of Total Probability

If events \( B_1, B_2, \dots, B_k \) are mutually exclusive and together cover the whole sample space, then

$$ P(A) = \sum_{i=1}^{k} P(B_i)\,P(A \mid B_i) $$

A tree diagram is the natural way to organise this: multiply along each branch, then add the branches that lead to \( A \).

Bayes' Theorem

Bayes' theorem lets us find \( P(B \mid A) \) when we know \( P(A \mid B) \):

$$ P(B_i \mid A) = \frac{P(B_i)\,P(A \mid B_i)}{\sum_{j} P(B_j)\,P(A \mid B_j)} $$

The numerator is one branch of the tree; the denominator is the total probability of \( A \). Key vocabulary:

• \( P(B_i) \) is the prior probability (before the evidence).
• \( P(A \mid B_i) \) is the likelihood of the evidence.
• \( P(B_i \mid A) \) is the posterior probability (after the evidence).

Worked Example 2: A Medical Test

Question: A disease affects 1% of a population. A test detects the disease 95% of the time when it is present, but also gives a positive result 5% of the time when it is absent. A person tests positive. What is the probability they actually have the disease?

Step 1: Write down the information. Let \( D \) = "has disease" and \( + \) = "tests positive".

• \( P(D) = 0.01 \), so \( P(D') = 0.99 \)
• \( P(+ \mid D) = 0.95 \)
• \( P(+ \mid D') = 0.05 \)

Step 2: Total probability of a positive test.

$$ P(+) = (0.01)(0.95) + (0.99)(0.05) = 0.0095 + 0.0495 = 0.059 $$

Step 3: Apply Bayes' theorem.

$$ P(D \mid +) = \frac{0.0095}{0.059} \approx 0.161 $$

So there is only about a 16% chance the person has the disease, despite the "95% accurate" test.

Why so low? Imagine 10,000 people. About 100 have the disease and 95 of them test positive. Of the 9,900 healthy people, 5% (495) also test positive. Out of \( 95 + 495 = 590 \) positives, only 95 are genuine: \( \frac{95}{590} \approx 0.161 \). When a condition is rare, false positives can outnumber true positives.

Worked Example 3: Which Machine Made It?

Question: Machine A produces 60% of a factory's bulbs and 2% of its bulbs are defective. Machine B produces 40%, with 5% defective. A randomly chosen bulb is defective. Find the probability it came from machine B.

Step 1: Branches.

• \( P(A)P(\text{def} \mid A) = 0.6 \times 0.02 = 0.012 \)
• \( P(B)P(\text{def} \mid B) = 0.4 \times 0.05 = 0.020 \)

Step 2: Total probability of a defective bulb. \( 0.012 + 0.020 = 0.032 \).

Step 3: Bayes.

$$ P(B \mid \text{def}) = \frac{0.020}{0.032} = 0.625 $$

Although B makes fewer bulbs, it is responsible for 62.5% of the defective ones.

Common Mistakes

• Confusing \( P(A \mid B) \) with \( P(B \mid A) \). "Positive given disease" is not "disease given positive", as Example 2 shows.
• Forgetting to reduce the sample space in drawing-without-replacement questions.
• Leaving out a branch when computing the total probability in the denominator.
• Ignoring the prior. A small prior probability strongly affects the posterior.

Key Takeaways

• \( P(A \mid B) = \dfrac{P(A \cap B)}{P(B)} \): restrict attention to the outcomes in \( B \).
• For dependent events, \( P(A \cap B) = P(A)P(B \mid A) \).
• The law of total probability adds up all the branches that lead to an event.
• Bayes' theorem reverses a condition: posterior = (prior × likelihood) ÷ total probability.
• Tree diagrams or "imagine 10,000 people" make Bayes problems much easier to follow.

If Bayes' theorem questions or probability assignments feel confusing, send them to Amathify on WhatsApp or via the "Ask a Question" form, and our experts and PhD scholars will provide step-by-step solutions.