3.5 Distributions

3.5 Normal Distribution and Probability

Normal (Gaussian) Distribution

• A normal random variable $X\sim \mathcal{N}(\mu ,{\sigma }^2)$ has probability-density function (PDF) $$f(x)=\frac{1}{\sqrt{2\pi {\sigma }^2}}\exp [-\frac{(x-\mu {)}^2}{2{\sigma }^2}],-\infty <x<\infty .$$
• Standardisation: the $z$-score
  $z=\frac{x-\mu }{\sigma }⟹Z\sim \mathcal{N}(0,1).$

Cumulative-Distribution Function (CDF) of a Gaussian

• The CDF gives the probability that $X$ does not exceed a threshold: $$F_X(x)=P(X\le x)=\frac{1}{2}[1+\mathrm{erf}(\frac{x-\mu }{\sigma \sqrt{2}}\left)\right],$$ where $\mathrm{erf}$ is the error function.
• In practice we consult the standard-normal table (or software) for $P(Z\le z)$ and convert $x\to z$ using the formula above.

---

Lognormal Distribution

• Some phenomena grow multiplicatively rather than additively. If
  $W\sim \mathcal{N}(\mu ,{\sigma }^2),X=\exp (W),$
  then $X$ is lognormal.
• PDF $$f(x)=\frac{1}{x\sigma \sqrt{2\pi }}\exp [-\frac{(\ln x-\mu {)}^2}{2{\sigma }^2}],x>0.$$
• Mean and variance $$\mathbb{E}[X]=e^{\mu +\frac{1}{2}{\sigma }^2},\mathrm{Var}[X]=(e^{{\sigma }^2}-1)e^{2\mu +{\sigma }^2}.$$

---

Gamma Distribution

• The gamma function generalises the factorial: $$\Gamma (r)={\int }_0^{\infty }{x}^{r-1}{e}^{-x}dx,r>0,\Gamma (r)=(r-1)\Gamma (r-1).$$
• With shape $k>0$ and rate $\lambda >0$, the gamma PDF is $$f(x)=\frac{{\lambda }^k}{\Gamma (k)}{x}^{k-1}{e}^{-\lambda x},x>0.$$
• Mean and variance $$\mu =\frac{k}{\lambda },{\sigma }^2=\frac{k}{{\lambda }^2}.$$ (For integer $k$ the gamma reduces to an Erlang; for $k=\frac{n}{2}$ and $\lambda =\frac{1}{2}$ it produces the ${\chi }^2$ distribution.)

---

Beta Distribution

• A continuous distribution bounded on $[0,1]$; useful for proportions.
• PDF with shape parameters $\alpha >0,\beta >0$: $$f(x)=\frac{\Gamma (\alpha +\beta )}{\Gamma (\alpha )\Gamma (\beta )}{x}^{\alpha -1}(1-x{)}^{\beta -1},0<x<1.$$
• Tuning $\alpha$ and $\beta$ changes the shape:
  • $\alpha =\beta =1$ → uniform;
  • $\alpha =\beta$ → symmetric about $x=0.5$;
  • $\alpha <1$ or $\beta <1$ → modes at the boundaries.
• Mean and variance $$\mu =\frac{\alpha }{\alpha +\beta },{\sigma }^2=\frac{\alpha \beta }{(\alpha +\beta {)}^2(\alpha +\beta +1)}.$$

---

Binomial Distribution

• A discrete model for $n$ independent Bernoulli trials (success with probability $p$, failure with $1-p$).
• Random variable $X$ = number of successes.
• Probability-mass function (PMF) $$P(X=k)=\left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k},k=0,1,\dots ,n.$$
• Mean and variance $$\mu =np,{\sigma }^2=np(1-p).$$

---