# Non-orthogonal bases

Monday, September 27, 2021

## Logistics

• Midterm 1
• General logistics of exams
• Assignment 4 assigned today or tomorrow

## What’s on the agenda for today?

• Last time: Orthobases

• Things work in infinite dimension too with orthobases (without thinking too much)
• Last little bit to prove: isomorphism to $\ell_2$
• Today: What happens with non-orthogonal bases

• Wednesday September 29, 2021: Least-square regression

• Reading: Romberg, lecture notes 7

## Recap - Orthobases in infinite dimension

• The following properties of an orthonormal set $\set{e_i}_{i\geq 1}$ are equivalent:

1. Finite linear combinations of elements in $\calH$ are dense in $\calH$;
2. If $v\in\calH$ and $\forall j\geq 1$ $\dotp{v}{e_j}=0$ then $v=0$;
3. If $v\in\calH$ then $\sum_{i=1}^n\dotp{v}{e_i}e_i$ converges to $v$ (in the norm $\norm{\cdot}$) as $n\to\infty$;
4. If $v\in\calH$ and $\forall k$ $a_k\eqdef \dotp{v}{e_k}$ then $\norm{v}^2=\sum_{k\geq 1}\abs{a_k}^2$.
• This does not say anything about the existence of such nice orthornormal set

• A space is separable if it contains a countable dense subset.
• Separability is the key property to deal with sequences instead of collections

• Any separable Hilbert space has an orthonormal basis.
• Most useful Hilbert spaces are separable! We won’t worry about non-separable Hilbert spaces

• Any separable Hilbert space is isomorphic to $\ell_2$

## Non-orthogonal bases in finite dimension

• Let $\set{v_i}_{i=1}^n$ be a linearly independent set in a Hilbert space $\calH$ of dimension $n$. Then, for any $x\in\calH$, $x=\sum_{i=1}^n\alpha_iv_i$ for some $\bfalpha\in\bbR^n$. In addition, there exists $A,B>0$ such that $A\norm[2]{\bfalpha}^2 \leq \norm[\calH]{x}^2\leq B\norm[2]{\bfalpha}^2$
• Inequality is tight on both sides

• For orthobases, $A=B=1$

• Interpretation:

• The values of $A$ and $B$ govern the stability of the representation
• Examples

## Non-orthogonal bases in finite dimension: dual basis

• Recall from orthobases:
• perfectly stable representation $A=B=1$
• Efficient computation of representations: $\alpha_i=\dotp{x}{v_i}$
• For any $x\in\calH$ with basis $\set{v_i}_{i=1}^n$we have $x=\sum_{i=1}^n\alpha_iv_i\qquad\text{with}\qquad\bfalpha = \matG^{-1}\mat{c}{\dotp{x}{v_1}\\\dotp{x}{v_2}\\\vdots\\\dotp{x}{v_n}}$ There also exists a basis $\set{\tilde{v}_i}_{i=1^n}$ such that $\alpha_i=\dotp{x}{\tilde{v}_i}$

## Non-orthogonal bases in infinite dimension

• $\set{v_i}_{i=1}^\infty$ is a Riesz basis for Hilbert space $\calH$ if $\text{cl}(\text{span}(\set{v_i}_{i=1}^\infty))=\calH$ and there exists $A,B>0$ such that $A\sum_{i=1}^\infty\alpha_i^2\leq \norm[\calH]{\sum_{i=1}^n\alpha_iv_i}^2\leq B\sum_{i=1}^\infty\alpha_i^2$ uniformly for all sequences $\set{\alpha_i}_{i\geq 1}$ with $\sum_{i\geq 1}\alpha_i^2<\infty$.
• In infinite dimension, the existence of $A,B>0$ is not automatic.

• Examples

## Non-orthogonal bases in finite dimension: dual basis

• Computing expansion on Riesz basis not as simple in infinite dimension: Gram matrix is “infinite”

• The Grammiam is a linear operator $\calG:\ell_2(\bbZ)\to\ell_2(\bbZ): \bfx\mapsto \bfy\text{ with }[\calG(\bfx)]_n\eqdef y_n=\sum_{\ell=-\infty^\infty}\dotp{v_\ell}{v_n}x_\ell$

• Fact: there exists another linear operator $\calH:\ell_2(\bbZ)\to\ell_2(\bbZ)$ such that

$\calH(\calG(\bfx)) = \bfx$ We can replicate what we did in finite dimension.