The Mathematics of ECE

Let $\mathrm{\Omega }\ne \mathrm{\varnothing }$ be a sample space. The class of subsets of $\mathrm{\Omega }$ that constitutes events satisfies the following axioms:

1. $\mathrm{\Omega }$ is an event;
2. For $\{{\mathcal{A}}_i{\}}_{i\ge 1}^{\mathrm{\infty }}$ some events in $\mathrm{\Omega }$, ${\cup }_{i=1}^{\mathrm{\infty }}{\mathcal{A}}_i$ is an event;
3. For every event $\mathcal{A}$ in $\mathrm{\Omega }$, ${\mathcal{A}}^c$ is an event.

Let $\mathrm{\Omega }\ne \mathrm{\varnothing }$ be a sample space and $\mathcal{F}$ a class of events satisfying the axioms for events. A probability rule is a function $\mathbb{P}:\mathcal{F}\to {\mathbb{R}}^{+}$ such that:

1. ${\mathbb{P}}_{}\left(\mathrm{\Omega }\right)=1$;
2. For every $\mathcal{A}\subset \mathcal{F}$ we have ${\mathbb{P}}_{}\left(\mathcal{A}\right)\ge 0$;
3. For any disjoint events in $\mathcal{F}$ $\{{\mathcal{A}}_i{\}}_{i=1}^{\mathrm{\infty }}$ we have ${\mathbb{P}}_{}\left({\cup }_{i=1}^{\mathrm{\infty }}{\mathcal{A}}_i\right)=\sum _{i=1}^{\mathrm{\infty }}{\mathbb{P}}_{}\left({\mathcal{A}}_i\right)$.

Let $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$ be a probability space. If ${\mathbb{P}}_{}\left(\mathcal{B}\right)>0$, the conditional probability of event $\mathcal{A}$ given event $\mathcal{B}$ is given by ${\mathbb{P}}_{}\left(\mathcal{A}|\mathcal{B}\right)={\mathbb{P}}_{}(\mathcal{A}\cap \mathcal{B})/{\mathbb{P}}_{}\left(\mathcal{B}\right)$.

Let $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$ be a probability space and $\mathcal{A},\mathcal{B}$ with non-zero probability. Then, $${\mathbb{P}}_{}\left(\mathcal{A}|\mathcal{B}\right)={\mathbb{P}}_{}\left(\mathcal{B}|\mathcal{A}\right)\frac{{\mathbb{P}}_{}\left(\mathcal{A}\right)}{{\mathbb{P}}_{}\left(\mathcal{B}\right)}.$$

Let $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$ be a probability space. Event $\mathcal{A}$ is independent of event $\mathcal{B}$ if ${\mathbb{P}}_{}(\mathcal{A}\cap \mathcal{B})={\mathbb{P}}_{}\left(\mathcal{A}\right){\mathbb{P}}_{}\left(\mathcal{B}\right)$.

For $n>2$, the events ${\{{\mathcal{A}}_i\}}_{i=1}^n$ are independent if $\mathrm{\forall }\mathcal{S}\subset [1;n]$ such that $|\mathcal{S}|\ge 2$, $${\mathbb{P}}_{}\left({\cap }_{i\in \mathcal{S}}{\mathcal{A}}_i\right)=\prod _{i\in \mathcal{S}}{\mathbb{P}}_{}\left({\mathcal{A}}_i\right)$$

Let $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$ be a probability space. A random variable $X$ is a function $X:\mathrm{\Omega }\to \mathbb{R}$.

1. $X$ might be undefined or infinite on a subset of zero probability.
2. $\{\omega \in \mathrm{\Omega }:X(\omega )\le x\}$ must be an event for all $x\in \mathbb{R}$
3. For a finite set of random variables ${\{X_i\}}_{i=1}^n$, the set $\{\omega :X_1(\omega )\le x_1,\cdots ,X_n(\omega )\le x_n\}$ must be an event for all ${\{x_i\}}_{i=1}^n$

Let $(\mathrm{\Omega },\mathcal{F},\mathbb{P})$ be a probability space and $X$ a random variable. The CDF of $X$ is the function $$F_X:\mathbb{R}\to [0,1]:x\mapsto {\mathbb{P}}_{}(\omega \in \mathrm{\Omega }:X(\omega )\le x)\triangleq {\mathbb{P}}_{}(X\le x)$$

Let $X$ be a random variable with PMF $P_X$. Then ${\mathbb{E}}_{}\left[X\right]\triangleq \sum _{x\in \mathcal{X}}x{P}_X(x)$.

Let $X$ be a random variable with PDF $p_X$. Then ${\mathbb{E}}_{}\left[X\right]\triangleq {\int }_{x\in \mathcal{X}}x{p}_X(x)dx$.

Let $X$ be a random variable. The $m$th moment of $X$ is ${\mathbb{E}}_{}\left[X^m\right]$.

The variance is the second centered moment $\text{Var}\left(X\right)\triangleq {\mathbb{E}}_{}[(X-{\mathbb{E}}_{}\left[X\right]{)}^2]={\mathbb{E}}_{}\left[X^2\right]-{{\mathbb{E}}_{}\left[X\right]}^2$.