\[ %% % Add your macros here; they'll be included in pdf and html output. %% \newcommand{\R}{\mathbb{R}} % reals \newcommand{\E}{\mathbb{E}} % expectation \renewcommand{\P}{\mathbb{P}} % probability \DeclareMathOperator{\logit}{logit} \DeclareMathOperator{\logistic}{logistic} \DeclareMathOperator{\SE}{SE} \DeclareMathOperator{\sd}{sd} \DeclareMathOperator{\var}{var} \DeclareMathOperator{\cov}{cov} \DeclareMathOperator{\cor}{cor} \DeclareMathOperator{\Normal}{Normal} \DeclareMathOperator{\MVN}{MVN} \DeclareMathOperator{\LogNormal}{logNormal} \DeclareMathOperator{\Poisson}{Poisson} \DeclareMathOperator{\Beta}{Beta} \DeclareMathOperator{\Binom}{Binomial} \DeclareMathOperator{\Gam}{Gamma} \DeclareMathOperator{\Exp}{Exponential} \DeclareMathOperator{\Cauchy}{Cauchy} \DeclareMathOperator{\Unif}{Unif} \DeclareMathOperator{\Dirichlet}{Dirichlet} \DeclareMathOperator{\Wishart}{Wishart} \DeclareMathOperator{\StudentsT}{StudentsT} \DeclareMathOperator{\Weibull}{Weibull} \newcommand{\given}{\;\vert\;} \]

Rules of probability

Peter Ralph

9 November – Advanced Biological Statistics

Probability: review and notation

Probability rules:

Probabilities are proportions: \(\hspace{2em} 0 \le \P\{A\} \le 1\)
Everything: \(\hspace{2em} \P\{ \Omega \} = 1\)
Complements: \(\hspace{2em} \P\{ \text{not } A\} = 1 - \P\{A\}\)
Disjoint events: If \(\hspace{2em} \P\{A \text{ and } B\} = 0\) then \(\hspace{2em} \P\{A \text{ or } B\} = \P\{A\} + \P\{B\}\).
Independence: \(A\) and \(B\) are independent iff \(\P\{A \text{ and } B\} = \P\{A\} \P\{B\}\).
Conditional probability: \[\P\{A \given B\} = \frac{\P\{A \text{ and } B\}}{ \P\{B\} }\]

Bayes’ rule

A consequence is

\[\P\{B \given A\} = \frac{\P\{B\} \P\{A \given B\}}{ \P\{A\} } .\]

In “Bayesian statistics”:

\(B\): possible model
\(A\): data
\(\P\{B\}\): prior weight on model \(B\)
\(\P\{A \given B\}\): likelihood of data under \(B\)
\(\P\{B\} \P\{A \given B\}\): posterior weight on \(B\)
\(\P\{A\}\): total sum of posterior weights

Example:

Coin #1 comes up heads 75% of the time, and coin #2 only comes up heads 25% of the time. We grab one coin at random, flip it 10 times, and get 6 Heads. What’s the probability that the coin we grabbed is coin #1?

\(B\): possible model (I have the 75% coin)
\(A\): data (6 Heads out of 10 flips)
\(\P\{B\}\): prior weight on model \(B\) (how likely I think it is I got the 75% coin)
\(\P\{A \given B\}\): likelihood of data under \(B\) (chance of getting 6 H with 10 flips with the 75% coin)
\(\P\{B\} \P\{A \given B\}\): posterior weight on \(B\) (combined probability of getting 75% coin and then getting 6 H out of 10 flips)
\(\P\{A\}\): total sum of posterior weights (the sum of the last thing over all possible coins)

\(\P\{B\} = 1/2\): (chance of getting 75% coin)
\(\P\{A \given B\} = \binom{10}{6} 0.75^6 \times 0.25^4 = 0.146\): (chance of getting 6 H with 10 flips with the 75% coin)
\(\P\{B\} \P\{A \given B\} = 1/2 \times 0.146 = 0.073\): posterior weight on \(B\) (combined probability of getting 75% coin and then getting 6 H out of 10 flips)
\(\P\{\text{not }B\} \P\{A \given \text{not }B\} = 1/2 \times \binom{10}{6} 0.25^6 \times 0.75^4 = 0.0081\): (same thing but with the other coin)
\(\P\{A\} = 0.073 + 0.0081= 0.081\): total sum of posterior weights (the sum of posterior weights over all possible coins)

\[\begin{aligned} \P\{B \given A\} &= \P\{\text{grabbed 75% coin}\given\text{6 heads from 10 flips}\} \\ &= \frac{\P\{B\} \P\{A \given B\}}{ \P\{A\} } \\ &= \frac{0.073}{0.073 + 0.0081} \\ &= 0.9 \end{aligned}\]