Vector spaces

Dr. Matthieu R Bloch

Monday, August 30, 2021

Logistics

Self assessment
- Due today (Monday August 30, 2021) (+48 hour grace period if forfeiting bonus)
- Open everything, collaboration widely encouraged, ask questions on Piazza
- Upload on Gradescope
Graduate teaching assistants + office hours
- Information to be confirmed, but the hope is to offer office hours every day
- Some will be online to help Q an QSZ sections
Sign up for piazza! (180 enrolled @ 1:45pm) Piazza
Check out the course slides: https://bloch.ece.gatech.edu/teaching/ece7750fa21/
Resources
- Gilbert Strang, Linear Algebra and Its Applications for linear algebra
- Papoulis and Pilai, Probability, Random Variables and Stochastic Processes for probabilities
- Review session tonight in TSRB auditorium (https://bloch.ece.gatech.edu/teaching/ece7750fa21/)

Last time:
- Playing with linear basis expansions (and why it matters for ML)
- We didn’t talk about splines: see the next homework? (it’s not that critical)
- Most importantly, I hope you got a feel for how the course will run
Today (and probably Wednesday): we’ll be formal about linear basis expansion
- Vector spaces (in particular of sequences or functions)
- Inner product, norm
- Hilbert spaces (and why they matter)
Definitions, propositions, proofs and examples!

You are probably used to working in $\bbR^n$ (review session tonight!)… we’ll deal with weirder objects
A vector space $\calV$ over a field $\bbK$ consists of a set $\calV$ of vectors, a closed addition rule $+$ and a closed scalar multiplication $\cdot$ such that 8 axioms are satisfied:
1. $\forall x,y\in\calV$ $x+y=y+x$ (commutativity)
2. $\forall x,y,z\in\calV$ $x+(y+z)=(x+y)+z$ (associativity)
3. $\exists 0\in\calV$ such that $\forall x\in\calV$ $x+0=x$ (identity element)
4. $\forall x\in\calV$ $\exists y\in\calV$ such that $x+y=0$ (inverse element)
5. $\forall x\in\calV$ $1\cdot x= x$
6. $\forall \alpha, \beta\in\bbK$ $\forall x\in\calV$ $\alpha\cdot(\beta\bfx)=(\alpha\cdot\beta)\cdot\bfx$ (associativity)
7. $\forall \alpha, \beta\in\bbK$ $\forall x\in\calV$ $(\alpha+\beta)x = \alpha x+\beta x$ (distributivity)
8. $\forall \alpha\in\bbK$ $\forall x,y\in\calV$ $\alpha(x+y) = \alpha x+\alpha y$ (distributivity)
- $0\in\calV$ is unique
- Every $x\in\calV$ has a unique inverse
- $0\cdot x = 0$
- The inverse of $x\in\calV$ is $(-1)\cdot x\eqdef -x$

\[ \calV_1\eqdef \left\{\mat{ccc}{x_1&\cdots&x_n}^\intercal:\set{x_i}_{i=1}^n\in\bbR^n\right\} \] How do we make $\calV_1$ a vector space?
\[ \calV_2\eqdef \left\{f:[a,b]\to\bbR:f \text{ continuous and bounded}\right\} \] How do we make $\calV_2$ a vector space?
\[ \calV_3\left\{\mat{ccc}{x_1&\cdots&x_n}^\intercal:\set{x_i}_{i=1}^n\in\bbF_2^n\right\} \] How do we make $\calV_3$ a vector space?
\[ \calV_4\eqdef \left\{f:[a,b]\to\bbR:f \text{ continuous and less than 2}\right\} \] How do we make $\calV_4$ a vector space?

A subset $\calW$ of a vector space $\calV$ is a vector subspace if $\forall x,y\in\calW\forall \lambda,\mu\in\bbK$ $\lambda x+\mu y \in\calW$
If $\calW$ is a vector subspace of a vector space $\calV$, $\calW$ is a vector space.

$\calV_1=\bbR^5$, $\calW_1\eqdef\set{x\in\calV_1: x_4=0,x_5=0}$ Is $\calW_1$ a vector subspace?
$\calV_2=\bbR^5$, $\calW_2\eqdef\set{x\in\calV_1: x_4=1,x_5=0}$ Is $\calW_2$ a vector subspace?
$\calV_3=\calC([0,1])$, $\calW_3\eqdef\set{\text{polynomials of degree at most $p$}}$ Is $\calW_3$ a vector subspace?
$\calV_4=\bbR^n$, $\calW_4\eqdef\set{x:x\text{ has not more than 5 non-zero components}}$ Is $\calW_4$ a vector subspace?

Let $\set{v_i}_{i=1}^n$ be a set of vectors in a vector space $\calV$.
For $\set{a_i}_{i=1}^n\in\bbK^n$, $\sum_{i=1}^na_iv_i$ is called a linear combination of the vectors $\set{v_i}_{i=1}^n$.
The span of the vectors $\set{v_i}_{i=1}^n$ is the set \[ \text{span}(\set{v_i}_{i=1}^n)\eqdef \{\sum_{i=1}^na_iv_i:\set{a_i}_{i=1}^n\in\bbK^n\} \]
The span of the vectors $\set{v_i}_{i=1}^n\in\calV^n$ is a vector subspace of $\calV$.

Subspaces in $\bbR^3$
$\calM=\set{b_0(t-k):k\in\set{0,1,2,3}}$ where $b_0(t)=\indic{0\leq t \leq 1}$. What is the span of $\calM$?