Probability rules:

0. Probabilities are proportions: $0\le \mathbb{P}\{A\}\le 1$

1. Everything: $\mathbb{P}\{\mathrm{\Omega }\}=1$

2. Complements: $\mathbb{P}\{\text{not }A\}=1-\mathbb{P}\{A\}$

3. Disjoint events: If $\mathbb{P}\{A\text{ and }B\}=0$ then $\mathbb{P}\{A\text{ or }B\}=\mathbb{P}\{A\}+\mathbb{P}\{B\}$.

4. Independence: $A$ and $B$ are independent iff $\mathbb{P}\{A\text{ and }B\}=\mathbb{P}\{A\}\mathbb{P}\{B\}$.

5. Conditional probability: $$\mathbb{P}\{A|B\}=\frac{\mathbb{P}\{A\text{ and }B\}}{\mathbb{P}\{B\}}$$

Bayes’ rule

A consequence is

$$\mathbb{P}\{B|A\}=\frac{\mathbb{P}\{B\}\mathbb{P}\{A|B\}}{\mathbb{P}\{A\}}.$$

Example:

Coin #1 comes up heads 75% of the time, and coin #2 only comes up heads 25% of the time. We grab one coin at random, flip it 10 times, and get 6 Heads. What’s the probability that the coin we grabbed is coin #1?

• $\mathbb{P}\{B\}=1/2$: (chance of getting 75% coin)
• $\mathbb{P}\{A|B\}=(\genfrac{}{}{0ex}{}{10}{6}){0.75}^6\times {0.25}^4=0.146$: (chance of getting 6 H with 10 flips with the 75% coin)
• $\mathbb{P}\{B\}\mathbb{P}\{A|B\}=1/2\times 0.146=0.073$: posterior weight on $B$ (combined probability of getting 75% coin and then getting 6 H out of 10 flips)
• $\mathbb{P}\{\text{not }B\}\mathbb{P}\{A|\text{not }B\}=1/2\times (\genfrac{}{}{0ex}{}{10}{6}){0.25}^6\times {0.75}^4=0.0081$: (same thing but with the other coin)
• $\mathbb{P}\{A\}=0.073+0.0081=0.081$: total sum of posterior weights (the sum of posterior weights over all possible coins)

$$\begin{array}{rl}\mathbb{P}\{B|A\}& =\mathbb{P}\{\text{grabbed 75\% coin}|\text{6 heads from 10 flips}\}\\ & =\frac{\mathbb{P}\{B\}\mathbb{P}\{A|B\}}{\mathbb{P}\{A\}}\\ & =\frac{0.073}{0.073+0.0081}\\ & =0.9\end{array}$$