Non-orthogonal bases

What’s on the agenda for today?

Last time: Orthobases
- Things work in infinite dimension too with orthobases (without thinking too much)
- Last little bit to prove: isomorphism to $ℓ_{2}$
Today: What happens with non-orthogonal bases
Wednesday September 29, 2021: Least-square regression
Reading: Romberg, lecture notes 7

The following properties of an orthonormal set ${e_{i}}_{i \geq 1}$ are equivalent:
1. Finite linear combinations of elements in $H$ are dense in $H$ ;
2. If $v \in H$ and $\forall j \geq 1$ ${⟨ v, e_{j} ⟩}_{} = 0$ then $v = 0$ ;
3. If $v \in H$ then $\sum_{i = 1}^{n} {⟨ v, e_{i} ⟩}_{} e_{i}$ converges to $v$ (in the norm ${‖ \cdot ‖}_{}$ ) as $n \to \infty$ ;
4. If $v \in H$ and $\forall k$ $a_{k} ≜ {⟨ v, e_{k} ⟩}_{}$ then ${‖ v ‖}_{}^{2} = \sum_{k \geq 1} {| a_{k} |}^{2}$ .
This does not say anything about the existence of such nice orthornormal set
A space is separable if it contains a countable dense subset.
Separability is the key property to deal with sequences instead of collections
Any separable Hilbert space has an orthonormal basis.
Most useful Hilbert spaces are separable! We won’t worry about non-separable Hilbert spaces
Any separable Hilbert space is isomorphic to $ℓ_{2}$

Let ${v_{i}}_{i = 1}^{n}$ be a linearly independent set in a Hilbert space $H$ of dimension $n$ . Then, for any $x \in H$ , $x = \sum_{i = 1}^{n} α_{i} v_{i}$ for some $α \in R^{n}$ . In addition, there exists $A, B > 0$ such that $A {‖ α ‖}_{2}^{2} \leq {‖ x ‖}_{H}^{2} \leq B {‖ α ‖}_{2}^{2}$
Inequality is tight on both sides
For orthobases, $A = B = 1$
Interpretation:
- The values of $A$ and $B$ govern the stability of the representation
Examples

Recall from orthobases:
- perfectly stable representation $A = B = 1$
- Efficient computation of representations: $α_{i} = {⟨ x, v_{i} ⟩}_{}$
For any $x \in H$ with basis ${v_{i}}_{i = 1}^{n}$ we have $x = \sum_{i = 1}^{n} α_{i} v_{i} with α = G^{- 1} [\begin{matrix} {⟨ x, v_{1} ⟩}_{} \\ {⟨ x, v_{2} ⟩}_{} \\ ⋮ \\ {⟨ x, v_{n} ⟩}_{} \end{matrix}]$ There also exists a basis ${{\tilde{v}}_{i}}_{i = 1^{n}}$ such that $α_{i} = {⟨ x, {\tilde{v}}_{i} ⟩}_{}$

${v_{i}}_{i = 1}^{\infty}$ is a Riesz basis for Hilbert space $H$ if $cl (span ({v_{i}}_{i = 1}^{\infty})) = H$ and there exists $A, B > 0$ such that $A \sum_{i = 1}^{\infty} α_{i}^{2} \leq {‖ \sum_{i = 1}^{n} α_{i} v_{i} ‖}_{H}^{2} \leq B \sum_{i = 1}^{\infty} α_{i}^{2}$ uniformly for all sequences ${α_{i}}_{i \geq 1}$ with $\sum_{i \geq 1} α_{i}^{2} < \infty$ .
In infinite dimension, the existence of $A, B > 0$ is not automatic.
Examples

Computing expansion on Riesz basis not as simple in infinite dimension: Gram matrix is “infinite”
The Grammiam is a linear operator $G : ℓ_{2} (Z) \to ℓ_{2} (Z) : x \mapsto y with [G (x)]_{n} ≜ y_{n} = \sum_{ℓ = - \infty^{\infty}} {⟨ v_{ℓ}, v_{n} ⟩}_{} x_{ℓ}$
Fact: there exists another linear operator $H : ℓ_{2} (Z) \to ℓ_{2} (Z)$ such that

$H (G (x)) = x$ We can replicate what we did in finite dimension.