Orthobases

Dr. Matthieu R Bloch

Monday, September 20, 2021

Logistics

  • Assigments 1&2: solution posted for #1, coming today for #2
  • Assignment 3
    • Hard deadline on Tuesday September 21, 2021 (exception!)
    • This is what matters most Midterm
  • Midterm 1
    • In-class exam during regular class time - 75 minutes
    • Topics: everything up to and including today’s coverage
    • Open notes (slides+HW solution + Prof. Romberg’s notes) - no electronics - no textbooks
    • Prof. Bloch office hours: Tuesday September 21, 2021 8am on https://bluejeans.com/9058052289

What’s on the agenda for today?

  • Last time: More on orthobases
    • Orthobases: things work in infinite dimension too
    • Examples!
  • Monday September 27, 2021: non orthogonal bases
  • Reading: Romberg, lecture notes 5 and 6

Orthobases

  • A collection of vectors \(\set{v_i}_{i=1}^n\) in a finite dimensional Hilbert space \(\calH\) is an orthobasis if 1) \(\text{span}(\set{v_i}_{i=1}^n)=\calH\); 2) \(\forall i\neq j\in\intseq{1}{n}\,v_i\perp v_j\); 3) \(\forall i\in\intseq{1}{n} \,\norm{v_i}=1\).

  • If the last condition is not met, this is just called an orthogonal basis

  • Orthobases are useful because we can write \(x=\sum_{i=1}^n\dotp{x}{v_i}v_i\) (what happens in a non-orthonormal basis?)

  • We would like to extend this idea to infinite dimension and happily write \(x=\sum_{i=1}^\infty\dotp{x}{v_i}v_i\)

    • We have to be a bit careful
    • With a little bit of machinery, this works: separability + completeness

What we know works in infinite dimension

  • Consider a Hilbert space \(\calH\). The following notions we encountered work without problems:

    • Norm induced by inner product
    • Cauchy-Schwartz inequality
    • Orthogonality + Pythagorean theorem + projection theorem

  • A collection of vectors \(\set{v_i}_{i\in\calI}\) in a Hilbert space \(\calV\) is orthonormal if

    1. \(\forall i\neq j\in\intseq{1}{n}\,v_i\perp v_j\);
    2. \(\forall i\in\intseq{1}{n} \,\norm{v_i}=1\).

  • We do not even require \(\calI\) to be countable at this stage.

  • We are not saying anything about this being a basis

Towards orthobases in infinite dimension

  • Let \(c\in\calH\) and let \(\set{x_n}_{n\geq 1}\) be a sequence of vectors in \(\calH\) that converges to \(x^*\in\calH\). Then \(\lim_{n\to\infty}\dotp{x_n}{c} = \dotp{x^*}{c}\)
  • Let \(\set{v_n}_{n\geq 1}\) be sequence of orthonormal vectors in \(\calH\). Then for any \(x\in\calH\), \(\sum_{n=1}^\infty\abs{\dotp{x}{v_n}}^2\leq\norm{x}^2\).

  • The following properties of an orthonormal set \(\set{e_i}_{i\geq 1}\) are equivalent:

    1. Finite linear combinations of elements in \(\calH\) are dense in \(\calH\);
    2. If \(v\in\calH\) and \(\forall j\geq 1\) \(\dotp{v}{e_j}=0\) then \(v=0\);
    3. If \(v\in\calH\) then \(\sum_{i=1}^n\dotp{v}{e_i}e_i\) converges to \(v\) (in the norm \(\norm{\cdot}\)) as \(n\to\infty\);
    4. If \(v\in\calH\) and \(\forall k\) \(a_k\eqdef \dotp{v}{e_k}\) then \(\norm{v}^2=\sum_{k\geq 1}\abs{a_k}^2\).

  • This does not say anything about the existence of such nice orthornormal set

Separable space

  • A space is separable if it contains a countable dense subset.

  • Separability is the key property to deal with sequences instead of collections

  • Any separable Hilbert space has an orthonormal basis.

  • Most useful Hilbert spaces are separable! We won’t worry about non-separable Hilbert spaces

  • Key take away for separable Hilbert spaces

    • \(x=\sum_{i=1}^\infty\dotp{x}{v_i}v_i\) is perfectly well defined for an orthonormal basis
    • Parseval’s identity tell us that \(\norm{x}^2=\sum_{k\geq 1}\abs{\dotp{x}{v_k}}^2\).
    • It looks like we don’t need to even worry about the nature of \(\calH\) and only think about coefficients \(\dotp{x}{v_i}\)
  • Any separable Hilbert space is isomorphic to \(\ell_2\)

Examples

  • Piecewise constant approximation
  • Sampling