The Mathematics of ECE

Analysis - Sequences in normed vector spaces

Friday, September 3, 2021

Logistics

The Mathematics of ECE
- No session on Monday September 06, 2021
Websites
- Piazza for course: sign-up
- Website for slides: https://bloch.ece.gatech.edu/teaching/MofECEfa21/
Today’s class: Analysis
- Sequences
- Some topology
- Notion of convergence

To talk about sequences, series, and convergence, one needs to be able to measure “closeness”
We will study this in the concepts of normed vector spaces (less abstract and very useful)
A vector space \(\calV\) over a field \(\bbK\) consists of a set \(\calV\) of vectors, a closed addition rule \(+\) and a closed scalar multiplication \(\cdot\) such that 8 axioms are satisfied:
1. \(\forall x,y\in\calV\) \(x+y=y+x\) (commutativity)
2. \(\forall x,y,z\in\calV\) \(x+(y+z)=(x+y)+z\) (associativity)
3. \(\exists 0\in\calV\) such that \(\forall x\in\calV\) \(x+0=x\) (identity element)
4. \(\forall x\in\calV\) \(\exists y\in\calV\) such that \(x+y=0\) (inverse element)
5. \(\forall x\in\calV\) \(1\cdot x= x\)
6. \(\forall \alpha, \beta\in\bbK\) \(\forall x\in\calV\) \(\alpha\cdot(\beta\bfx)=(\alpha\cdot\beta)\cdot\bfx\) (associativity)
7. \(\forall \alpha, \beta\in\bbK\) \(\forall x\in\calV\) \((\alpha+\beta)x = \alpha x+\beta x\) (distributivity)
8. \(\forall \alpha\in\bbK\) \(\forall x,y\in\calV\) \(\alpha(x+y) = \alpha x+\alpha y\) (distributivity)
This is pretty dense, but the rules of algebra are very natural
- we can add and scale vectors

A vector space provides an algebraic structure
We are missing a topological structure to measure length and distance
A norm on a vector space \(\calV\) over \(\bbR\) is a function \(\norm{\cdot}:\calV\to\bbR\) that satisfies:
- Positive definiteness: \(\forall x\in\calV\) \(\norm{x}\geq 0\) with equality iff \(x=0\)
- Homogeneity: \(\forall x\in\calV\) \(\forall\alpha\in\bbR\) \(\norm{\alpha x}=\abs{\alpha}\norm{x}\)
- Subadditivity: \(\forall x,y\in\calV\) \(\norm{x+y}\leq \norm{x}+\norm{y}\)
\(\norm{x}\) measures a length, \(\norm{x-y}\) measures a distance
\(\bfx\in\bbR^d\qquad\norm[0]{\bfx}\eqdef\card{\set{i:x_i\neq 0}}\quad\norm[1]{\bfx}\eqdef\sum_{i=1}^d\abs{x_i}\quad \norm[2]{\bfx}\eqdef\sqrt{\sum_{i=1}^d x_i^2}\)

The study of convergence, limits, continuity requires some notions of topology
For \(x\in\calV\), the open ball centered at \(x\) with radius \(\epsilon>0\) is \[ \calB(x,\epsilon)\eqdef\set{v\in\calV:\norm{x-v}<\epsilon} \]
Let \(\calP\) be a subset of a normed space \(\calV\). Then \(p\in\calP\) is an interior point of \(\calP\) if \[ \exists \epsilon>0 \text{ such that } \calB(p,\epsilon)\subset\calP \] The collection of all interior points of \(\calP\) is the interior of \(\calP\), denoted \(\mathring{P}\)
As set \(\calP\) is open if \(\mathring{\calP}=\calP\)

A point \(x\in\calV\) is a closure point of a set \(\calP\) if \[ \forall \epsilon>0\exists p\in\calP\text{ such that } x\in\calB(p,\epsilon) \] The collection of all closure points is called the closure of \(\calP\) and denoted \(\bar{P}\).
As set \(\calP\) is closed if \(\bar{\calP}=\calP\)
The complement of an open set is closed, the complement of a closed set is open.
The intersection of a finite number of open set is open; the union of an arbitrary collection of open sets is open.
The union of a finite number of closed set is closed; the intersection of an arbitrary collection of closed sets is closed.

A sequence \(\set{x_i}_{i\in\bbN}\) in a vector space \(\calV\) is a map \(\bbN\to\calV\). The image of \(i\) is the \(i\)th term of the sequence and denoted by \(x_i\).
A sequence can also be indexed by a subset of \(\bbN\)
A sequence \(\set{x_i}_{i\in\bbN}\) in a normed vector space \(\calV\) converges to a vector \(x\in\calV\) if and only if \[ \forall \epsilon>0 \exists N_\epsilon\in\bbN \text{ s.t. } \forall n\geq N_\epsilon \,\norm{x_n-x}\leq \epsilon \]
This is equivalent to saying that the sequence of real numbers \(\set{\norm{x_n-x}}\) converges to zero.
We often write \(x_n\to_{n\to\infty} x\)