Dr. Matthieu R Bloch

Wednesday September 15, 2021

**Assigment 2**- Hard deadline on Thursday September 16, 2021 (hard)

**Assignment 3**- Posted yesterday night
- Due Monday September 20, 2021 (soft)
- Hard deadline on Tuesday September 21, 2021 (
**exception!**)

**Midterm 1**- As of Wednesday, September 15, 2021 10:18am, Surveillance Incidence Positivity at 0.53%
- In-class exam during regular class time - 75 minutes
- Topics: everything up to and including today’s coverage
- Open notes (slides+HW solution + Prof. Romberg’s notes) - no electronics

**Last time**: Orthogonality principle**Key take-way:**Closest vector problem has nice properties:*orthogonality principle*- This can be solve nicely computationally!
**Questions?**

**Today:***Orthobases:*some theory and more about infinite dimensional Hilbert spaces*Examples!*

**Monday September 20, 2021**: non orthogonal bases**Reading:**Romberg, lecture notes 5 and 6

- A collection of vectors \(\set{v_i}_{i=1}^n\) in a
*finite*dimensional Hilbert space \(\calH\) is an*orthobasis*if 1) \(\text{span}(\set{v_i}_{i=1}^n)=\calH\); 2) \(\forall i\neq j\in\intseq{1}{n}\,v_i\perp v_j\); 3) \(\forall i\in\intseq{1}{n} \,\norm{v_i}=1\). If the last condition is not met, this is just called an

*orthogonal*basisOrthobases are useful because we can write \(x=\sum_{i=1}^n\dotp{x}{v_i}v_i\) (what happens in a non-orthonormal basis?)

We would like to extend this idea to infinite dimension and happily write \(x=\sum_{i=1}^\infty\dotp{x}{v_i}v_i\)

- We have to be a bit careful
- With a little bit of machinery, this
*works*: separability + completeness

Consider a Hilbert space \(\calH\). The following notions we encountered work without problems:

- Norm induced by inner product
- Cauchy-Schwartz inequality

- Orthogonality + Pythagorean theorem + projection theorem

A collection of vectors \(\set{v_i}_{i\in\calI}\) in a Hilbert space \(\calV\) is

*orthonormal*if- \(\forall i\neq j\in\intseq{1}{n}\,v_i\perp v_j\);
- \(\forall i\in\intseq{1}{n} \,\norm{v_i}=1\).

We do not even require \(\calI\) to be countable at this stage.

We are not saying anything about this being a basis

- Let \(c\in\calH\) and let \(\set{x_n}_{n\geq 1}\) be a sequence of vectors in \(\calH\) that converges to \(x^*\in\calH\). Then \(\lim_{n\to\infty}\dotp{x_n}{c} = \dotp{x^*}{c}\)

- Let \(\set{v_n}_{n\geq 1}\) be sequence of orthonormal vectors in \(\calH\). Then for any \(x\in\calH\), \(\sum_{n=1}^\infty\abs{\dotp{x}{v_n}}^2\leq\norm{x}^2\).
The following properties of an orthonormal set \(\set{e_i}_{i\geq 1}\) are equivalent:

- Finite linear combinations of elements in \(\calH\) are dense in \(\calH\);
- If \(v\in\calH\) and \(\forall j\geq 1\) \(\dotp{v}{e_j}=0\) then \(v=0\);
- If \(v\in\calH\) then \(\sum_{i=1}^n\dotp{v}{e_i}e_i\) converges to \(v\) (in the norm \(\norm{\cdot}\)) as \(n\to\infty\);
- If \(v\in\calH\) and \(\forall k\) \(a_k\eqdef \dotp{v}{e_k}\) then \(\norm{v}^2=\sum_{k\geq 1}\abs{a_k}^2\).

This does not say anything about the existence of such nice orthornormal set

- A space is separable if it contains a
*countable*dense subset. Separability is the key property to deal with sequences instead of collections

- Any separable Hilbert space has an orthonormal basis.
Most useful Hilbert spaces are separable! We won’t worry about non-separable Hilbert spaces

**Key take away for separable Hilbert spaces**- \(x=\sum_{i=1}^\infty\dotp{x}{v_i}v_i\) is perfectly well defined for an orthonormal basis
- Parseval’s identity tell us that \(\norm{x}^2=\sum_{k\geq 1}\abs{\dotp{x}{v_k}}^2\).
- It looks like we don’t need to even worry about the nature of \(\calH\) and only think about coefficients \(\dotp{x}{v_i}\)

- Any separable Hilbert space is isomorphic to \(\ell_2\)

- Piecewise constant approximation
- Sampling