The Mathematics of ECE

Analysis - Sequences and series

Monday September 13, 2021

Analysis: roadmap

Last time: review of concepts from topology
- Key concepts: normed vector spaces, open, closed sets, convergence
- Key results: algebraic structure not enough to talk about convergence
- Useful machine learning
Today: More on convergence, sequences and series
- Key concepts: Cauchy sequences, convergence
- Useful in signal processing, machine learning
Website for slides: https://bloch.ece.gatech.edu/teaching/MofECEfa21/

A sequence \(\set{x_i}_{i\in\bbN}\) in vector space \(\calV\) is Cauchy if \[ \forall \epsilon>0\quad\exists N\in\bbN^*\text{ such that }\forall m,n>N \norm{x_m-x_n}\leq\epsilon. \]
Intuition: the points get closer to each others
Every converging sequence is Cauchy
Not every Cauchy sequence is converging.
Requesting that every Cauchy sequence converges amounts to requiring a complete vector space (ECE 7750)
A Cauchy sequence is bounded
We will review a very special situation: sequence in \(\bbR\), for which there is a total order

\(\bbR\) is a vector space and everything discussed earlier applies
An increasing bounded sequence \(\set{x_n}_{n\geq 1}\) in \(\bbR\) converges to its least upper bound.
Let \(\set{x_n}_{n\geq 1}\) be a bounded sequence in \(\bbR\). Then \[ \limsup_{n\to\infty} x_n = \lim_{n\to\infty}\left(\sup\set{x_k:k\geq n}\right)\qquad \liminf_{n\to\infty} x_n = \lim_{n\to\infty}\left(\inf\set{x_k:k\geq n}\right) \]
A sequence \(\set{x_n}_{n\geq 1}\) converges to \(x\in\bbR\) if and only if \[ \limsup_{n\to\infty} x_n =\liminf_{n\to\infty} x_n \]
A Cauchy sequence \(\set{x_n}_{n\geq 1}\) in \(\bbR\) converges. This proves that \(\bbR\) is complete.

Let \(\set{v_n}_{n\geq 1}\) be sequence in a normed vector space. The series \(\sum_{n=1}^\infty v_n\) converges to if and only if \(\lim_{m\to\infty}\sum_{n=1}^m v_n=s\).
If the series \(\sum_{n=1}^\infty v_n\) converges, then \(\lim_{n\to\infty}v_n = 0\).
This is not a sufficient condition for convergence
There are other tests of convergence
Let \(\set{a_n}_{n\geq 1}\) be real valued and non-negative. The series \(\sum_{n=1}^\infty a_n\) converges if and only if the partial sums \(s_m\eqdef\sum_{n=1}^m a_n\) are bounded.
Let \(\set{a_n}_{n\geq 1}\) be real valued and non-negative.
1. If there exists \(C\in(0;1)\) and \(N\) such that \(a_{n+1}\leq C a_n\) for all \(n>N\), then \(\sum_{n=1}^\infty a_n\) converges
2. If there exists \(C>1\) and \(N\) such that \(a_{n+1}\geq C a_n\) for all \(n>N\), then \(\sum_{n=1}^\infty a_n\) diverges.
Let \(f:\bbR\to\bbR\) be continuous and positive and decreasing for all \(x\geq 1\). Then \(\sum_{n=1}^\infty f(n)\) converges if and only if \(\int_{1}^\infty f(x) dx\) converges.