The Beta Distribution

If \[P \sim \text{Beta}(a,b)\] then \(P\) has probability density \[p(\theta) = \frac{ \theta^{a-1} (1 - \theta)^{b-1} }{ B(a,b) } . \]

• Takes values between 0 and 1.

• If \(U_{(1)} < U_{(2)} < \cdots < U_{(n)}\) are sorted, independent \(\text{Unif}[0,1]\) then \(U_{(k)} \sim \text{Beta}(k, n-k+1)\).

• Mean: \(a/(a+b)\).

• Larger \(a+b\) is more tightly concentrated (like \(1/\sqrt{a+b}\))

Beta-Binomial Bayesian analysis

If \[\begin{aligned} P &\sim \text{Beta}(a,b) \\ Z &\sim \text{Binom}(n,P) , \end{aligned}\] then by Bayes’ rule: \[\begin{aligned} \P\{ P = \theta \given Z = z\} &= \frac{\P\{Z = z \given P = \theta \} \P\{P = \theta\}}{\P\{Z = z\}} \\ &= \frac{ \binom{n}{z}\theta^z (1-\theta)^{n-z} \times \frac{\theta^{a-1}(1-\theta)^{b-1}}{B(a,b)} }{ \text{(something)} } \\ &= \text{(something else)} \times \theta^{a + z - 1} (1-\theta)^{b + n - z - 1} . \end{aligned}\]

“Miraculously”
(the Beta is the conjugate prior for the Binomial), \[\begin{aligned} (P \given Z = z) \sim \text{Beta}(a+z, b+n-z) . \end{aligned}\]

Exercise

Simulate:

1. One thousand “random coins” whose probabilities are drawn from a \(\Beta(5,5)\) distribution. (rbeta()) Make a histogram of these probabilities.

2. Flip each coin ten times and record the number of heads. (rbinom())

3. Make a histogram of the probabilities of those coins that got exactly 3 heads, and compare to the first histogram.