Hilbert spaces

Dr. Matthieu R Bloch

Wednesday September 08, 2021

Logistics

  • Assigment 2
    • Posted Wednesday September 08, 2021
    • Due Tuesday September 14, 2021
  • Assignment 3
    • Posted Tuesday September 14, 2021
    • Due Monday September 20, 2021
  • Midterm 1

What’s on the agenda for today?

  • Last time:
    • normed vector spaces, inner product vector spaces (pre-Hilbert)
    • Key take-way: we can’t do geometry without an inner product
    • Key concepts: norm, inner product
    • Questions?
  • Today: Hilbert spaces
    • More on vector spaces: Cauchy-Schwartz, orthogonality, angles
    • Orthogonal projections
  • Monday September 13, 2021: orthobases (and non orthogonal ones)

Induced norm

  • In an inner product space, an inner product induces a norm \(\norm{x} \eqdef \sqrt{\dotp{x}{x}}\)

  • A norm \(\norm{\cdot}\) is induced by an inner product on \(\calV\) iff \(\forall x,y\in\calV\) \(\norm{x}^2+\norm{y}^2 = \frac{1}{2}\left(\norm{x+y}^2+\norm{x-y}^2\right)\) If this is the case, the inner product is given by the polarization identity \[\dotp{x}{y}=\frac{1}{2}\left(\norm{x}^2+\norm{y}^2-\norm{x-y}^2\right)\]

  • Induced norm have some nice additional properties

  • An induced norm satisfies \(\forall x,y\in\calV\) \(\norm{x+y}\leq \norm{x}+\norm{y}\)

  • An inner product satisfies \(\forall x,y\in\calV\) \(\dotp{x}{y}^2\leq\dotp{x}{x}\dotp{y}{y}\)

Orthogonality

  • In the following \(\calV\) is an inner product space with induced norm \(\norm{\cdot}\)
  • The angle between two non-zero vectors \(x,y\in\calV\) is \[ \cos\theta \eqdef \frac{\dotp{x}{y}}{\norm{x}\norm{y}} \]

  • Two vectors \(x,y\in\calV\) are orthogonal if \(\dotp{x}{y}=0\). We write \(x\perp y\) for simplicity.

    A vector \(x\in\calV\) is orthogonal to a set \(\calS\subset\calV\) if \(\forall s\in\calS\) \(\dotp{x}{s}=0\). We write \(x\perp \calS\) for simplicity.
  • If \(x\perp y\) then \(\norm{x+y}^2=\norm{x}^2+\norm{y}^2\)

  • Inner product spaces have almost all the properties of \(\bbR^n\)!

Hilbert spaces

  • In infinite dimensions, things are a little bit tricky. What does the following mean? \[ x(t) = \sum_{n=1}^\infty \alpha_n\psi_n(t) \]

  • We need to define a notion of convergence, e.g., \[ \lim_{N\to\infty}\norm{x(t)-\sum_{n=1}^N \alpha_n\psi_n(t)}=0 \]

  • Problems can still arise if “points are missing”; we avoid this by introducing the notion of completeness

  • A inner product space \(\calV\) is complete if every Cauchy sequence converges, i.e., for every \(\set{x_i}_{i\geq1}\) in \(\calV\) \[ \lim_{\min(m,n)\to\infty}\norm{x_m-x_n}=0\Rightarrow \lim_{n\to\infty}x_n = x^*\in\calV. \]

  • We won’t worry too much about proving that spaces are complete

  • A complete normed vector space is a Banach space; a complete inner product space is a Hilbert space

Orthogonality principle

  • Let \(\calH\) be a Hilbert space with induced norm \(\dotp{\cdot}{\cdot}\) and induced norm \(\norm{\cdot}\) ; let \(\calT\) be subspace of \(\calH\)

  • For \(x\in\calH\), what is the closest point of \(\hat{x}\in\calT\)? How do we solve

    \[ \min_{y\in\calT}\norm{x-y} \]

  • This problem has a unique solution given by the orthogonality principle

  • Let \(\calX\) be a pre-Hilbert space, \(\calT\) be a subspace of \(\calX\), and \(x\in\calX\).

    • If there exists a vector \(m^*\in\calT\) such that \(\forall m\in\calT\) \(\norm{x-m^*}\leq \norm{x-m}\), then \(m^*\) is unique.
    • A necessary and sufficient condition for \(m^*\in\calT\) to be a unique minimizer is that the error \(x-m^*\) be orthogonal to \(\calT\).

  • This doesn’t say that \(m^*\) exists!

  • Let \(\calH\) be a Hilbert space, \(\calT\) be a closed subspace of \(\calX\), and \(x\in\calX\).

    • There exists a unique vector \(m^*\in\calT\) such that \(\forall m\in\calT\) \(\norm{x-m^*}\leq \norm{x-m}\).
    • A necessary and sufficient condition for \(m^*\in\calT\) to be a unique minimizer is that the error \(x-m^*\) be orthogonal to \(\calT\)

    • Computing approximations

    • The orthogonality principle gives us a procedure for computing the closest point

    • Let \(\calH\) be a Hilbert space, \(\calT\) be a subspace of \(\calH\) with dimension \(n\), and \(x\in\calH\). Let \(\set{e_i}_{i=1}^n\) be a basis for \(\calT\). Then the projection \(\hat{x}\) of \(x\) onto \(\calT\) is \[ \hat{x} = \sum_{i=1}^n\alpha_i e_i \] where \(\bfalpha\eqdef\mat{c}{\alpha_1&\cdots&\alpha_n}^\intercal\) is the solution of \(\bfG\bfalpha=\bfb\) with \(\bfG\) the Grammiam matrix of the basis and \(\bfb\) the coordinates of \(x\) on the basis.