Probability basics
Probability puts a number on uncertainty. It is the foundation for every inference statistics makes from samples.
What probability means
The probability of an event is the long-run proportion of times it happens if the random process is repeated many times. A fair coin lands heads with probability \(0.5\): in thousands of flips, close to half are heads, even though short runs can be streaky. Probabilities range from \(0\) (impossible) to \(1\) (certain).
Sample spaces and events
The sample space \(S\) lists all possible outcomes. Rolling a die: \(S = \{1, 2, 3, 4, 5, 6\}\). An event is a set of outcomes, such as 'even' \(= \{2, 4, 6\}\). When outcomes are equally likely, \(P(A) = \frac{\text{outcomes in } A}{\text{outcomes in } S}\), so \(P(\text{even}) = \frac{3}{6} = \frac{1}{2}\).
P(A) = \frac{|A|}{|S|}The complement rule
The complement of \(A\), written \(A^c\), is 'not \(A\)'. Since one or the other must happen, \(P(A^c) = 1 - P(A)\). This is often the easiest route: the probability of at least one six in four rolls is \(1 - \left(\frac{5}{6}\right)^4 \approx 0.518\).
P(A^c) = 1 - P(A)The addition rule
For 'A or B': \(P(A \cup B) = P(A) + P(B) - P(A \cap B)\). We subtract the overlap so it is not counted twice. If \(A\) and \(B\) cannot happen together (mutually exclusive), the overlap is \(0\) and the probabilities simply add. Drawing a card that is a heart or a king: \(\frac{13}{52} + \frac{4}{52} - \frac{1}{52} = \frac{16}{52}\).
P(A \cup B) = P(A) + P(B) - P(A \cap B)Venn diagrams and tables
Venn diagrams show events as overlapping circles, making the addition rule visual. Two-way tables of counts let you read probabilities directly: divide a cell by the grand total for a joint probability, or by a row or column total for a conditional probability.
The law of large numbers
As the number of trials grows, the observed proportion of an event gets closer to its true probability. This does not mean outcomes 'balance out' in the short run. After five heads in a row, the next flip of a fair coin is still \(0.5\); believing otherwise is the gambler's fallacy.
Simulation
When a probability is hard to calculate, simulate: use random numbers to imitate the process many times and count how often the event occurs. Simulations are used to model queues, weather, financial risk, and games. With enough repetitions, simulated proportions approximate true probabilities.
Try it yourself
A bag has 3 red and 5 blue marbles. Find \(P(\text{red})\) and \(P(\text{not red})\). Answer: \(\frac{3}{8}\) and \(\frac{5}{8}\). Then find the probability of rolling a sum of 7 with two dice. Answer: 6 of the 36 equally likely outcomes, so \(\frac{1}{6}\).
Key takeaways
Probability is a long-run proportion between 0 and 1. With equally likely outcomes, count favourable outcomes over total outcomes. Use the complement rule for 'at least one' problems and the addition rule (subtracting overlaps) for 'or'. The law of large numbers describes the long run, not short-run balancing.
Flashcard
Long-run proportion of occurrence
Check your understanding
What is the probability of rolling at least one six in two rolls of a fair die?