Random Variables and Distributions

A random variable is a measurable function from a sample space to the reals, and its distribution assigns probabilities to the values it takes. This note builds up the machinery of PMFs, PDFs, CDFs, and the standard distributions used throughout MIE286. See Primer on Probability for the underlying axioms.

Sample Space, Events, Random Variables

• Sample space $\Omega$: the set of all outcomes of an experiment.
• Event: a subset $A\subseteq \Omega$ assigned a probability.
• Probability measure $P$ satisfies the Kolmogorov axioms: $P(\Omega )=1$, $P(A)\ge 0$, and countable additivity $P({\bigcup }_i{A}_i)={\sum }_{i}P(A_i)$ for disjoint $A_i$.

A random variable (RV) $X:\Omega \to \mathbb{R}$ maps outcomes to numbers so we can reason about distributions instead of raw outcomes. Formally $\{X\le x\}$ must be an event for every $x$.

Discrete vs Continuous

Type	Values	Described by	Normalization
Discrete	countable set	PMF $p_X(x)=P(X=x)$	${\sum }_x{p}_X(x)=1$
Continuous	interval of $\mathbb{R}$	PDF $f_X(x)\ge 0$, $P(X=x)=0$	${\int }_{-\infty }^{\infty }{f}_X(x)dx=1$

PMF, PDF, CDF

The cumulative distribution function (CDF) works for both types:

$F_X(x)=P(X\le x)$

• Discrete: $F_X(x)={\sum }_{t\le x}{p}_X(t)$, a right-continuous step function.
• Continuous: $F_X(x)={\int }_{-\infty }^x{f}_X(t)dt$, so $f_X(x)=F_X^{\prime }(x)$ wherever the derivative exists.

$F_X$ is nondecreasing, right-continuous, with ${\lim }_{x\to -\infty }{F}_X=0$ and ${\lim }_{x\to \infty }{F}_X=1$. Interval probabilities: $P(a<X\le b)=F_X(b)-F_X(a)$.

PDF is not a probability

$f_X(x)$ can exceed $1$; only its integral over an interval is a probability. The product $f_X(x)dx$ approximates $P(x<X\le x+dx)$.

Key Distributions

Distribution	PMF / PDF	Mean	Variance	MGF $M_X(t)$
Bernoulli($p$)	$p^x(1-p{)}^{1-x}$, $x\in \{0,1\}$	$p$	$p(1-p)$	$1-p+p{e}^t$
Binomial($n,p$)	$\left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k}$	$np$	$np(1-p)$	$(1-p+p{e}^t{)}^n$
Geometric($p$)	$(1-p{)}^{k-1}p$, $k\ge 1$	$1/p$	$(1-p)/p^2$	$\frac{p{e}^t}{1-(1-p)e^t}$
Poisson($\lambda$)	$e^{-\lambda }{\lambda }^k/k!$	$\lambda$	$\lambda$	$e^{\lambda (e^t-1)}$
Uniform($a,b$)	$\frac{1}{b-a}$ on $[a,b]$	$\frac{a+b}{2}$	$\frac{(b-a{)}^2}{12}$	$\frac{e^{tb}-e^{ta}}{t(b-a)}$
Exponential($\lambda$)	$\lambda e^{-\lambda x}$, $x\ge 0$	$1/\lambda$	$1/{\lambda }^2$	$\frac{\lambda }{\lambda -t}$, $t<\lambda$
Normal($\mu ,{\sigma }^2$)	$\frac{1}{\sqrt{2\pi }\sigma }{e}^{-(x-\mu {)}^2/2{\sigma }^2}$	$\mu$	${\sigma }^2$	$e^{\mu t+{\sigma }^2{t}^2/2}$

Reading the moment generating function (MGF) and its uses is covered in Expectation and Moments.

• Binomial counts successes in $n$ independent Bernoulli trials.
• Geometric counts trials until the first success and is the discrete memoryless distribution.
• Poisson models rare event counts and is the $n\to \infty$, $np\to \lambda$ limit of the binomial.
• Exponential models waiting times and is memoryless: $P(X>s+t\mid X>s)=P(X>t)$.
• Normal is the limit in the Central Limit Theorem.

Standardizing the normal

If $X\sim \mathcal{N}(\mu ,{\sigma }^2)$ then $Z=(X-\mu )/\sigma \sim \mathcal{N}(0,1)$, reducing every normal probability to a standard normal table lookup.

Example: Poisson as a Binomial Limit

Let $X\sim \text{Bin}(n,p)$ with $np=\lambda$ fixed as $n\to \infty$. Then

$P(X=k)=\left(\genfrac{}{}{0ex}{}{n}{k}\right){\left(\frac{\lambda }{n}\right)}^k{\left(1-\frac{\lambda }{n}\right)}^{n-k}⟶e^{-\lambda }\frac{{\lambda }^k}{k!}.$

The falling factorial $n(n-1)\cdots (n-k+1)/n^k\to 1$ and $(1-\lambda /n{)}^n\to e^{-\lambda }$. So counts of many rare independent events are approximately Poisson.

Transformations of Random Variables

Given $Y=g(X)$, the distribution of $Y$ follows from that of $X$.

• CDF method (general): compute $F_Y(y)=P(g(X)\le y)$ directly, then differentiate.
• Change of variables for monotonic differentiable $g$: $f_Y(y)=f_X(g^{-1}(y))\left|\frac{d}{dy}{g}^{-1}(y)\right|$.
• Non-monotonic $g$: split the domain into monotonic pieces and sum their contributions.

Support and Jacobian errors

When transforming, always track the new support (range of $g$) and never drop the absolute value of the Jacobian. Forgetting either produces a “density” that fails to integrate to $1$.