# Orthogonality principle and Orthobases

Monday September 13, 2021

## Logistics

• Assigment 2
• Due Tuesday September 14, 2021 (soft)
• Due Thursday September 16, 2021 (hard)
• Assignment 3
• Posted Tuesday September 14, 2021
• Due Monday September 20, 2021 (soft)
• Midterm 1

## What’s on the agenda for today?

• Last time: Hilbert spaces

• Key take-way: we can avoid problems in infinite dimensions

• Key concepts: completeness is the property that we want

• Questions?

• Today:

• Orthogonality principle: computing closest points
• Orthobases: some theory and more about infinite dimensional Hilbert spaces
• Wednesday September 15, 2021: lots of examples

• Reading: Romberg, lecture notes 5 and 6

## Orthogonality principle

• Let $\calH$ be a Hilbert space with induced norm $\dotp{\cdot}{\cdot}$ and induced norm $\norm{\cdot}$ ; let $\calT$ be subspace of $\calH$

• For $x\in\calH$, what is the closest point of $\hat{x}\in\calT$? How do we solve $\min_{y\in\calT}\norm{x-y}$?

• This problem has a unique solution given by the orthogonality principle

• Let $\calX$ be a pre-Hilbert space, $\calT$ be a subspace of $\calX$, and $x\in\calX$.

If there exists a vector $m^*\in\calT$ such that $\forall m\in\calT$ $\norm{x-m^*}\leq \norm{x-m}$, then $m^*$ is unique.

$m^*\in\calT$ is a unique minimizer if and only if the error $x-m^*$ be orthogonal to $\calT$.
• This doesn’t say that $m^*$ exists!

• Let $\calH$ be a Hilbert space, $\calT$ be a closed subspace of $\calX$, and $x\in\calX$.

There exists a unique vector $m^*\in\calT$ such that $\forall m\in\calT$ $\norm{x-m^*}\leq \norm{x-m}$.

$m^*\in\calT$ is a unique minimizer if and only if the error $x-m^*$ be orthogonal to $\calT$

## Computing approximations

• The orthogonality principle gives us a procedure for computing the closest point

• Let $\calH$ be a Hilbert space, $\calT$ be a subspace of $\calH$ with dimension $n$, and $x\in\calH$. Let $\set{e_i}_{i=1}^n$ be a basis for $\calT$. Then the projection $\hat{x}$ of $x$ onto $\calT$ is $\hat{x} = \sum_{i=1}^n\alpha_i e_i$ where $\bfalpha\eqdef\mat{c}{\alpha_1&\cdots&\alpha_n}^\intercal$ is the solution of $\bfG\bfalpha=\bfb$ with $\bfG$ the Grammiam matrix of the basis and $\bfb$ the coordinates of $x$ on the basis.