Vector spaces

Logistics

Self assessment
- Due today (Monday August 30, 2021) (+48 hour grace period if forfeiting bonus)
- Open everything, collaboration widely encouraged, ask questions on Piazza
- Upload on Gradescope
Graduate teaching assistants + office hours
- Information to be confirmed, but the hope is to offer office hours every day
- Some will be online to help Q an QSZ sections
Sign up for piazza! (180 enrolled @ 1:45pm) Piazza
Check out the course slides: https://bloch.ece.gatech.edu/teaching/ece7750fa21/
Resources
- Gilbert Strang, Linear Algebra and Its Applications for linear algebra
- Papoulis and Pilai, Probability, Random Variables and Stochastic Processes for probabilities
- Review session tonight in TSRB auditorium (https://bloch.ece.gatech.edu/teaching/ece7750fa21/)

Last time:
- Playing with linear basis expansions (and why it matters for ML)
- We didn’t talk about splines: see the next homework? (it’s not that critical)
- Most importantly, I hope you got a feel for how the course will run
Today (and probably Wednesday): we’ll be formal about linear basis expansion
- Vector spaces (in particular of sequences or functions)
- Inner product, norm
- Hilbert spaces (and why they matter)
Definitions, propositions, proofs and examples!

You are probably used to working in $R^{n}$ (review session tonight!)… we’ll deal with weirder objects
A vector space $V$ over a field $K$ consists of a set $V$ of vectors, a closed addition rule $+$ and a closed scalar multiplication $\cdot$ such that 8 axioms are satisfied:
1. $\forall x, y \in V$ $x + y = y + x$ (commutativity)
2. $\forall x, y, z \in V$ $x + (y + z) = (x + y) + z$ (associativity)
3. $\exists 0 \in V$ such that $\forall x \in V$ $x + 0 = x$ (identity element)
4. $\forall x \in V$ $\exists y \in V$ such that $x + y = 0$ (inverse element)
5. $\forall x \in V$ $1 \cdot x = x$
6. $\forall α, β \in K$ $\forall x \in V$ $α \cdot (β x) = (α \cdot β) \cdot x$ (associativity)
7. $\forall α, β \in K$ $\forall x \in V$ $(α + β) x = α x + β x$ (distributivity)
8. $\forall α \in K$ $\forall x, y \in V$ $α (x + y) = α x + α y$ (distributivity)
- $0 \in V$ is unique
- Every $x \in V$ has a unique inverse
- $0 \cdot x = 0$
- The inverse of $x \in V$ is $(- 1) \cdot x ≜ - x$

$V_{1} ≜ {{[\begin{array}{ccc} x_{1} & \dots & x_{n} \end{array}]}^{⊺} : {x_{i}}_{i = 1}^{n} \in R^{n}}$ How do we make $V_{1}$ a vector space?
$V_{2} ≜ {f : [a, b] \to R : f continuous and bounded}$ How do we make $V_{2}$ a vector space?
$V_{3} {{[\begin{array}{ccc} x_{1} & \dots & x_{n} \end{array}]}^{⊺} : {x_{i}}_{i = 1}^{n} \in F_{2}^{n}}$ How do we make $V_{3}$ a vector space?
$V_{4} ≜ {f : [a, b] \to R : f continuous and less than 2}$ How do we make $V_{4}$ a vector space?

A subset $W$ of a vector space $V$ is a vector subspace if $\forall x, y \in W \forall λ, μ \in K$ $λ x + μ y \in W$
If $W$ is a vector subspace of a vector space $V$ , $W$ is a vector space.

$V_{1} = R^{5}$ , $W_{1} ≜ {x \in V_{1} : x_{4} = 0, x_{5} = 0}$ Is $W_{1}$ a vector subspace?
$V_{2} = R^{5}$ , $W_{2} ≜ {x \in V_{1} : x_{4} = 1, x_{5} = 0}$ Is $W_{2}$ a vector subspace?
$V_{3} = C ([0, 1])$ , $W_{3} ≜ {polynomials of degree at most p}$ Is $W_{3}$ a vector subspace?
$V_{4} = R^{n}$ , $W_{4} ≜ {x : x has not more than 5 non-zero components}$ Is $W_{4}$ a vector subspace?

Let ${v_{i}}_{i = 1}^{n}$ be a set of vectors in a vector space $V$ .
For ${a_{i}}_{i = 1}^{n} \in K^{n}$ , $\sum_{i = 1}^{n} a_{i} v_{i}$ is called a linear combination of the vectors ${v_{i}}_{i = 1}^{n}$ .
The span of the vectors ${v_{i}}_{i = 1}^{n}$ is the set $span ({v_{i}}_{i = 1}^{n}) ≜ {\sum_{i = 1}^{n} a_{i} v_{i} : {a_{i}}_{i = 1}^{n} \in K^{n}}$
The span of the vectors ${v_{i}}_{i = 1}^{n} \in V^{n}$ is a vector subspace of $V$ .

Subspaces in $R^{3}$
$M = {b_{0} (t - k) : k \in {0, 1, 2, 3}}$ where $b_{0} (t) = 1 {0 \leq t \leq 1}$ . What is the span of $M$ ?