The Mathematics of ECE

Logistics

Housekeeping
- Piazza for course: sign-up
- Slides available here
Today’s class: Probability & Statistics
- Key concepts: inequalities, law of large numbers, characteristic functions, moment generating functions
Next workshop: Friday September 16, 2022

Let $X$ be a real, non-negative random variable. Then for all $t > 0$ $P_{} (X \geq t) \leq \frac{E_{} [X]}{t}$
Drill: Prove Markov’s Inequality.
Let $X$ be a real random variable. Then for all $t > 0$ $P_{} (| X - E_{} [X] | \geq t) \leq \frac{{Var}_{} (X)}{t^{2}}$
Drill: Prove Chebyshev’s Inequality.
- Hint: Let $Y = (X - E_{} [X])^{2}$ , which is a real, non-negative random variable. From Markov’s Inequality…

Let ${X_{i}}_{i = 1}^{N}$ be independent and identically distributed (i.i.d.) with finite mean $μ = E_{} [X]$ . Then for all $ϵ > 0$ $lim_{N \to \infty} P_{} (| \frac{1}{N} \sum_{i = 1}^{N} X_{i} - μ | \geq ϵ) = 0$
Drill: Prove that, in the case of finite variance $σ^{2}$ of $X$ $P_{} (| \frac{1}{N} \sum_{i = 1}^{N} X_{i} - μ | \geq ϵ) \leq \frac{σ^{2}}{N ϵ^{2}}$
- Hint: Let ${\overset{―}{X}}_{N} = \frac{1}{N} \sum_{i = 1}^{N} X_{i}$ , which has mean $E_{} [{\overset{―}{X}}_{N}] = E_{} [X] = μ$ and variance ${Var}_{} ({\overset{―}{X}}_{N}) = \frac{1}{N} {Var}_{} (X) = \frac{σ^{2}}{N}$ . From Chebyshev’s Inequality…

When two independent random variables $X, Y$ are added to make a new random variable $Z = X + Y$ , the pdf of $Z$ is the convolution of those of $X, Y$ , i.e. $f_{Z} (z) = \int_{x} f_{X} (x) f_{Y} (z - x) d x$
When the sums grow to be of many terms, computing so many convolutions can be inconvenient. Where have we seen this before…
The characteristic function $φ_{X}$ of a random variable $X$ is defined as $φ_{X} (t) = E_{} [e^{j t X}]$
Because $E_{} [e^{j t X}] = \int_{x} f_{X} (x) e^{j t x} d x$ , the characteristic function is analogous to a Fourier transform of the pdf $f_{X}$ .
Drill: Show that for independent $X, Y$ and $Z = X + Y$ the characteristic function $φ_{Z} (t) = φ_{X} (t) φ_{Y} (t)$ (convolution in one domain is multiplication in the other).