Bayesian estimation of a proportion
Estimating a proportion is the classic Bayesian example. The beta distribution makes the updating especially clean.
The beta prior
For a proportion \(p\), a convenient prior is the beta distribution \(\text{Beta}(a, b)\), whose mean is \(\frac{a}{a + b}\). \(\text{Beta}(1, 1)\) is uniform, expressing no preference for any value. Think of \(a\) and \(b\) as 'prior successes' and 'prior failures'.
Conjugate updating
If the prior is \(\text{Beta}(a, b)\) and we observe \(k\) successes in \(n\) trials, the posterior is \(\text{Beta}(a + k, b + n - k)\). We simply add successes and failures to the prior counts. Priors with this property are called conjugate priors, and they make Bayesian calculations easy by hand.
\text{Beta}(a,b) \to \text{Beta}(a+k,\ b+n-k)Worked example
Start with a uniform prior \(\text{Beta}(1, 1)\) for a new website's click-through rate. In 50 visits there are 7 clicks. The posterior is \(\text{Beta}(8, 44)\), with mean \(\frac{8}{52} \approx 0.154\), slightly pulled toward the prior mean of 0.5 compared with the raw rate \(\frac{7}{50} = 0.14\). With more data, the pull of the prior fades.
Credible intervals
A 95% credible interval contains 95% of the posterior probability. Its interpretation is direct: given the data and prior, there is a 95% probability that \(p\) lies in this interval. For the website example, the interval is roughly 0.07 to 0.26. This is the statement people often mistakenly make about confidence intervals.
Posterior predictions
The posterior also predicts future data. The probability that the next visitor clicks is the posterior mean, about 0.154. Simulating many draws of \(p\) from the posterior and then simulating clicks gives a full predictive distribution for, say, the next 1,000 visits.
Comparing two proportions
For an A/B test, give each version its own beta posterior and compute the probability that B's rate exceeds A's, often by simulation. A statement like 'there is a 96% probability that design B converts better' is often easier for decision-makers to use than a p-value.
Shrinkage
Bayesian estimates pull extreme values toward the prior mean, which is called shrinkage. A new restaurant with one 5-star review is not truly perfect; a Bayesian rating with a sensible prior would rank it below a restaurant with 400 reviews averaging 4.8. Many rating systems use this idea.
Common mistakes
Forgetting to add the prior counts. Treating a credible interval and a confidence interval as computed the same way. Using an extreme prior that overwhelms modest data. Reporting the posterior mean without its uncertainty.
Key takeaways
Beta priors pair with binomial data: add successes and failures to the prior. The posterior mean blends prior and data, with data dominating as \(n\) grows. Credible intervals have the direct probability interpretation. Shrinkage protects against over-reacting to small samples.
Flashcard
A distribution for proportions
Check your understanding
With a Beta(2, 2) prior and 6 successes in 10 trials, the posterior is: