Orthogonality principle and Orthobases

Dr. Matthieu R Bloch

Monday September 13, 2021

Logistics

  • Assigment 2
    • Due Tuesday September 14, 2021 (soft)
    • Due Thursday September 16, 2021 (hard)
  • Assignment 3
    • Posted Tuesday September 14, 2021
    • Due Monday September 20, 2021 (soft)
  • Midterm 1

What’s on the agenda for today?

  • Last time: Hilbert spaces

    • Key take-way: we can avoid problems in infinite dimensions

    • Key concepts: completeness is the property that we want

    • Questions?

  • Today:

    • Orthogonality principle: computing closest points
    • Orthobases: some theory and more about infinite dimensional Hilbert spaces

  • Wednesday September 15, 2021: lots of examples

  • Reading: Romberg, lecture notes 5 and 6

Orthogonality principle

  • Let \(\calH\) be a Hilbert space with induced norm \(\dotp{\cdot}{\cdot}\) and induced norm \(\norm{\cdot}\) ; let \(\calT\) be subspace of \(\calH\)

  • For \(x\in\calH\), what is the closest point of \(\hat{x}\in\calT\)? How do we solve \(\min_{y\in\calT}\norm{x-y}\)?

  • This problem has a unique solution given by the orthogonality principle

  • Let \(\calX\) be a pre-Hilbert space, \(\calT\) be a subspace of \(\calX\), and \(x\in\calX\).

    If there exists a vector \(m^*\in\calT\) such that \(\forall m\in\calT\) \(\norm{x-m^*}\leq \norm{x-m}\), then \(m^*\) is unique.

    \(m^*\in\calT\) is a unique minimizer if and only if the error \(x-m^*\) be orthogonal to \(\calT\).

  • This doesn’t say that \(m^*\) exists!

  • Let \(\calH\) be a Hilbert space, \(\calT\) be a closed subspace of \(\calX\), and \(x\in\calX\).

    There exists a unique vector \(m^*\in\calT\) such that \(\forall m\in\calT\) \(\norm{x-m^*}\leq \norm{x-m}\).

    \(m^*\in\calT\) is a unique minimizer if and only if the error \(x-m^*\) be orthogonal to \(\calT\)

Computing approximations

  • The orthogonality principle gives us a procedure for computing the closest point

  • Let \(\calH\) be a Hilbert space, \(\calT\) be a subspace of \(\calH\) with dimension \(n\), and \(x\in\calH\). Let \(\set{e_i}_{i=1}^n\) be a basis for \(\calT\). Then the projection \(\hat{x}\) of \(x\) onto \(\calT\) is \[ \hat{x} = \sum_{i=1}^n\alpha_i e_i \] where \(\bfalpha\eqdef\mat{c}{\alpha_1&\cdots&\alpha_n}^\intercal\) is the solution of \(\bfG\bfalpha=\bfb\) with \(\bfG\) the Grammiam matrix of the basis and \(\bfb\) the coordinates of \(x\) on the basis.