Random Variables

Definition

A random variable $X$ is a function that maps outcomes from a sample space $\Omega$ to real numbers:

\[X: \Omega \to \mathbb{R}\]

Discrete random variable: takes countable values (e.g., number of heads, word counts).

Continuous random variable: takes uncountable values in an interval (e.g., height, time).

Probability Mass Function (PMF)

For discrete random variables:

\[p(x) = P(X = x)\]

Properties:

• $0 \leq p(x) \leq 1$
• $\sum_x p(x) = 1$
• $P(X \in A) = \sum_{x \in A} p(x)$

Probability Density Function (PDF)

For continuous random variables:

\[f(x) \geq 0, \quad \int_{-\infty}^{\infty} f(x) dx = 1\]

The probability over an interval:

\[P(a \leq X \leq b) = \int_a^b f(x) dx\]

Important: $P(X = x) = 0$ for continuous variables (probability at a point is zero).

Cumulative Distribution Function (CDF)

The CDF gives the probability that $X$ is less than or equal to $x$:

\[F(x) = P(X \leq x)\]

For discrete variables:

\[F(x) = \sum_{t \leq x} p(t)\]

For continuous variables:

\[F(x) = \int_{-\infty}^x f(t) dt\]

Properties:

• $F(x)$ is non-decreasing
• $\lim_{x \to -\infty} F(x) = 0$, $\lim_{x \to \infty} F(x) = 1$
• $P(a < X \leq b) = F(b) - F(a)$
• PDF is the derivative: $f(x) = \frac{d}{dx} F(x)$

Survival Function

Complement of the CDF:

\[S(x) = P(X > x) = 1 - F(x)\]

Used in reliability analysis and survival analysis.

Quantiles and Percentiles

The $p$-th quantile (or $100p$-th percentile) is the value $x_p$ such that:

\[F(x_p) = p\]

Special quantiles:

• Median: $x_{0.5}$ (50th percentile)
• Quartiles: $x_{0.25}, x_{0.5}, x_{0.75}$ (25th, 50th, 75th percentiles)
• Interquartile range (IQR): $x_{0.75} - x_{0.25}$

Indicator Random Variables

For an event $A$, the indicator variable:

\[\mathbb{1}_A = \begin{cases} 1 & \text{if } A \text{ occurs} \\ 0 & \text{otherwise} \end{cases}\]

Useful for converting events into numeric quantities for expectation calculations.

Transformations of Random Variables

If $Y = g(X)$, the distribution of $Y$ can be derived from $X$.

For monotonic $g$:

\[f_Y(y) = f_X(g^{-1}(y)) \left| \frac{d}{dy} g^{-1}(y) \right|\]

The term $\lvert \frac{d}{dy} g^{-1}(y) \rvert$ is the Jacobian of the transformation.

Joint Distributions

For two random variables $X$ and $Y$:

Joint PMF/PDF: $p(x, y) = P(X = x, Y = y)$ or $f(x, y)$

Marginal distribution:

\[p_X(x) = \sum_y p(x, y) \quad \text{(discrete)}\] \[f_X(x) = \int_{-\infty}^{\infty} f(x, y) dy \quad \text{(continuous)}\]

Conditional Distributions

Conditional PMF/PDF:

\[p(y|x) = \frac{p(x, y)}{p(x)}, \quad p(x) > 0\] \[f(y|x) = \frac{f(x, y)}{f_X(x)}, \quad f_X(x) > 0\]

Independent Random Variables

$X$ and $Y$ are independent if:

\[p(x, y) = p_X(x) p_Y(y)\]

Equivalently: $p(y \mid x) = p_Y(y)$ (knowing $X$ gives no information about $Y$).