# Orthobases

Wednesday September 15, 2021

## Logistics

• Assigment 2
• Hard deadline on Thursday September 16, 2021 (hard)
• Assignment 3
• Posted yesterday night
• Due Monday September 20, 2021 (soft)
• Hard deadline on Tuesday September 21, 2021 (exception!)
• Midterm 1
• As of Wednesday, September 15, 2021 10:18am, Surveillance Incidence Positivity at 0.53%
• In-class exam during regular class time - 75 minutes
• Topics: everything up to and including today’s coverage
• Open notes (slides+HW solution + Prof. Romberg’s notes) - no electronics

## What’s on the agenda for today?

• Last time: Orthogonality principle

• Key take-way: Closest vector problem has nice properties: orthogonality principle
• This can be solve nicely computationally!
• Questions?
• Today:

• Orthobases: some theory and more about infinite dimensional Hilbert spaces
• Examples!
• Monday September 20, 2021: non orthogonal bases

• Reading: Romberg, lecture notes 5 and 6

## Orthobases

• A collection of vectors $\set{v_i}_{i=1}^n$ in a finite dimensional Hilbert space $\calH$ is an orthobasis if 1) $\text{span}(\set{v_i}_{i=1}^n)=\calH$; 2) $\forall i\neq j\in\intseq{1}{n}\,v_i\perp v_j$; 3) $\forall i\in\intseq{1}{n} \,\norm{v_i}=1$.
• If the last condition is not met, this is just called an orthogonal basis

• Orthobases are useful because we can write $x=\sum_{i=1}^n\dotp{x}{v_i}v_i$ (what happens in a non-orthonormal basis?)

• We would like to extend this idea to infinite dimension and happily write $x=\sum_{i=1}^\infty\dotp{x}{v_i}v_i$

• We have to be a bit careful
• With a little bit of machinery, this works: separability + completeness

## What we know works in infinite dimension

• Consider a Hilbert space $\calH$. The following notions we encountered work without problems:

• Norm induced by inner product
• Cauchy-Schwartz inequality
• Orthogonality + Pythagorean theorem + projection theorem
• A collection of vectors $\set{v_i}_{i\in\calI}$ in a Hilbert space $\calV$ is orthonormal if

1. $\forall i\neq j\in\intseq{1}{n}\,v_i\perp v_j$;
2. $\forall i\in\intseq{1}{n} \,\norm{v_i}=1$.
• We do not even require $\calI$ to be countable at this stage.

• We are not saying anything about this being a basis

## Towards orthobases in infinite dimension

• Let $c\in\calH$ and let $\set{x_n}_{n\geq 1}$ be a sequence of vectors in $\calH$ that converges to $x^*\in\calH$. Then $\lim_{n\to\infty}\dotp{x_n}{c} = \dotp{x^*}{c}$
• Let $\set{v_n}_{n\geq 1}$ be sequence of orthonormal vectors in $\calH$. Then for any $x\in\calH$, $\sum_{n=1}^\infty\abs{\dotp{x}{v_n}}^2\leq\norm{x}^2$.
• The following properties of an orthonormal set $\set{e_i}_{i\geq 1}$ are equivalent:

1. Finite linear combinations of elements in $\calH$ are dense in $\calH$;
2. If $v\in\calH$ and $\forall j\geq 1$ $\dotp{v}{e_j}=0$ then $v=0$;
3. If $v\in\calH$ then $\sum_{i=1}^n\dotp{v}{e_i}e_i$ converges to $v$ (in the norm $\norm{\cdot}$) as $n\to\infty$;
4. If $v\in\calH$ and $\forall k$ $a_k\eqdef \dotp{v}{e_k}$ then $\norm{v}^2=\sum_{k\geq 1}\abs{a_k}^2$.
• This does not say anything about the existence of such nice orthornormal set

## Separable space

• A space is separable if it contains a countable dense subset.
• Separability is the key property to deal with sequences instead of collections

• Any separable Hilbert space has an orthonormal basis.
• Most useful Hilbert spaces are separable! We won’t worry about non-separable Hilbert spaces

• Key take away for separable Hilbert spaces

• $x=\sum_{i=1}^\infty\dotp{x}{v_i}v_i$ is perfectly well defined for an orthonormal basis
• Parseval’s identity tell us that $\norm{x}^2=\sum_{k\geq 1}\abs{\dotp{x}{v_k}}^2$.
• It looks like we don’t need to even worry about the nature of $\calH$ and only think about coefficients $\dotp{x}{v_i}$
• Any separable Hilbert space is isomorphic to $\ell_2$

## Examples

• Piecewise constant approximation
• Sampling