Hilbert spaces

Logistics

Assigment 2
- Posted Wednesday September 08, 2021
- Due Tuesday September 14, 2021
Assignment 3
- Posted Tuesday September 14, 2021
- Due Monday September 20, 2021
Midterm 1
- Will make decision regarding modality by next Wednesday.
- Moved to take home exam if Surveillance Incidence Positivity >1
- https://health.gatech.edu/coronavirus/health-alerts

Last time:
- normed vector spaces, inner product vector spaces (pre-Hilbert)
- Key take-way: we can’t do geometry without an inner product
- Key concepts: norm, inner product
- Questions?
Today: Hilbert spaces
- More on vector spaces: Cauchy-Schwartz, orthogonality, angles
- Orthogonal projections
Monday September 13, 2021: orthobases (and non orthogonal ones)

In an inner product space, an inner product induces a norm ${‖ x ‖}_{} ≜ \sqrt{{⟨ x, x ⟩}_{}}$
A norm ${‖ \cdot ‖}_{}$ is induced by an inner product on $V$ iff $\forall x, y \in V$ ${‖ x ‖}_{}^{2} + {‖ y ‖}_{}^{2} = \frac{1}{2} ({‖ x + y ‖}_{}^{2} + {‖ x - y ‖}_{}^{2})$ If this is the case, the inner product is given by the polarization identity ${⟨ x, y ⟩}_{} = \frac{1}{2} ({‖ x ‖}_{}^{2} + {‖ y ‖}_{}^{2} - {‖ x - y ‖}_{}^{2})$
Induced norm have some nice additional properties
An induced norm satisfies $\forall x, y \in V$ ${‖ x + y ‖}_{} \leq {‖ x ‖}_{} + {‖ y ‖}_{}$
An inner product satisfies $\forall x, y \in V$ ${⟨ x, y ⟩}_{}^{2} \leq {⟨ x, x ⟩}_{} {⟨ y, y ⟩}_{}$

In the following $V$ is an inner product space with induced norm ${‖ \cdot ‖}_{}$
The angle between two non-zero vectors $x, y \in V$ is $\cos θ ≜ \frac{{⟨ x, y ⟩}_{}}{{‖ x ‖}_{} {‖ y ‖}_{}}$
Two vectors $x, y \in V$ are orthogonal if ${⟨ x, y ⟩}_{} = 0$ . We write $x ⊥ y$ for simplicity.
A vector $x \in V$ is orthogonal to a set $S \subset V$ if $\forall s \in S$ ${⟨ x, s ⟩}_{} = 0$ . We write $x ⊥ S$ for simplicity.
If $x ⊥ y$ then ${‖ x + y ‖}_{}^{2} = {‖ x ‖}_{}^{2} + {‖ y ‖}_{}^{2}$
Inner product spaces have almost all the properties of $R^{n}$ !

In infinite dimensions, things are a little bit tricky. What does the following mean? $x (t) = \sum_{n = 1}^{\infty} α_{n} ψ_{n} (t)$
We need to define a notion of convergence, e.g., $lim_{N \to \infty} {‖ x (t) - \sum_{n = 1}^{N} α_{n} ψ_{n} (t) ‖}_{} = 0$
Problems can still arise if “points are missing”; we avoid this by introducing the notion of completeness
A inner product space $V$ is complete if every Cauchy sequence converges, i.e., for every ${x_{i}}_{i \geq 1}$ in $V$ $lim_{min (m, n) \to \infty} {‖ x_{m} - x_{n} ‖}_{} = 0 \Rightarrow lim_{n \to \infty} x_{n} = x^{*} \in V .$
We won’t worry too much about proving that spaces are complete
A complete normed vector space is a Banach space; a complete inner product space is a Hilbert space

Let $H$ be a Hilbert space with induced norm ${⟨ \cdot, \cdot ⟩}_{}$ and induced norm ${‖ \cdot ‖}_{}$ ; let $T$ be subspace of $H$
For $x \in H$ , what is the closest point of $\hat{x} \in T$ ? How do we solve

$min_{y \in T} {‖ x - y ‖}_{}$
This problem has a unique solution given by the orthogonality principle
Let $X$ be a pre-Hilbert space, $T$ be a subspace of $X$ , and $x \in X$ .
- If there exists a vector $m^{*} \in T$ such that $\forall m \in T$ ${‖ x - m^{*} ‖}_{} \leq {‖ x - m ‖}_{}$ , then $m^{*}$ is unique.
- A necessary and sufficient condition for $m^{*} \in T$ to be a unique minimizer is that the error $x - m^{*}$ be orthogonal to $T$ .

This doesn’t say that $m^{*}$ exists!

Let $H$ be a Hilbert space, $T$ be a closed subspace of $X$ , and $x \in X$ .

There exists a unique vector $m^{*} \in T$ such that $\forall m \in T$ ${‖ x - m^{*} ‖}_{} \leq {‖ x - m ‖}_{}$ .
A necessary and sufficient condition for $m^{*} \in T$ to be a unique minimizer is that the error $x - m^{*}$ be orthogonal to $T$

The orthogonality principle gives us a procedure for computing the closest point
Let $H$ be a Hilbert space, $T$ be a subspace of $H$ with dimension $n$ , and $x \in H$ . Let ${e_{i}}_{i = 1}^{n}$ be a basis for $T$ . Then the projection $\hat{x}$ of $x$ onto $T$ is $\hat{x} = \sum_{i = 1}^{n} α_{i} e_{i}$ where $α ≜ {[\begin{matrix} α_{1} & \dots & α_{n} \end{matrix}]}^{⊺}$ is the solution of $G α = b$ with $G$ the Grammiam matrix of the basis and $b$ the coordinates of $x$ on the basis.