pre-Hilbert spaces

Dr. Matthieu R Bloch

Wednesday September 01, 2021

Logistics

  • Self assessment

    • Due today (firm deadline)
    • Upload on Gradescope

  • Assignment 2 coming

  • Graduate teaching assistants + office hours

    • Monday 1:00pm - 2:00pm: TSRB 423 (Shi-Yuan)
    • Tuesdays 12:00pm - 1:00pm: Tutorial Lab in Van Leer room C449 Cubicle D (Kayla)
    • Wednedays 5:30 pm - 6:30pm: virtual (Kayla)
    • Thursdays 10am - 11 am: Tutorial Lab in Van Leer room C449 Cubicle B (Kayla)
    • Friday 1:00pm - 2:00pm: TSRB 423 (Meng-Che)

  • Course slides: https://bloch.ece.gatech.edu/teaching/ece7750fa21/

What’s on the agenda for today?

  • Last time:

    • Formal introduction to vector spaces
    • Key take-way: understand that vectors are not just arrows in \(\bbR^3\) (functions)
    • Key concepts: subspaces, spans, linear combinations
    • Questions?

  • Today: pre-Hilbert spaces (inner product spaces)

    • More on vector spaces: linear independence, bases, dimension
    • Norm, inner product

  • Monday September 06, 2021: Hilbert spaces

Linear independence

  • Let \(\set{v_i}_{i=1}^n\) be a set of vectors in a vector space \(\calV\)

  • \(\set{v_i}_{i=1}^n\) is linearly independent (or the vectors \(\set{v_i}_{i=1}^n\) are linearly independent ) if (and only if) \[ \sum_{i=1}^na_iv_i = 0\Rightarrow \forall i\in\intseq{1}{n}\,a_i=0 \] Otherwise the set is (or the vectors are) linearly dependent.

  • \[ \calV\eqdef\bbR^3\quad \bfv_1=\mat{ccc}{2&1^0}^\intercal, \bfv_2=\mat{ccc}{1&1&0}^\intercal, \bfv_3=\mat{ccc}{1&2&0}^\intercal \]
  • \[ \calV\eqdef\calC([0;1])\quad v_1=\cos(2\pi t), v_2=\sin(2\pi t), v_3=2\cos(2\pi t +\frac{\pi}{3}) \]

  • Any set of linearly dependent vectors contains a subset of linearly independent vectors with the same span.

Bases

  • A basis of vector subspace \(\calW\) of a vector space \(\calV\) is a countable set of vectors \(\calB\) such that:

    1. \(\text{span}(\calB)=\calW\)
    2. \(\calB\) is linearly independent
  • If a non vector space \(\calV\neq \set{0}\) as a finite basis with \(n\in\bbN^*\) elements, \(n\) is called the dimension of \(\calV\), denoted \(\dim{\calV}\). If the basis has an infinite number of elements, the dimension is infinite
  • Any two bases for the same finite dimensional vector space contain the same number of elements.

  • You should be somewhat familiar with bases (at least in \(\bbR^n\)):

    • the representation of a vector on a basis is unique
    • every subspace has a basis
    • having a basis reduces the operations on vectors to operations on their components

  • Things sort of work in infinite dimensions, but we have to be bit more careful

  • \(\calV=\bbR^n\), \(\calV=\set{f:f\text{ is periodic with period }2\pi}\)

Norm

  • The properties of vector space seen thus far provide an algebraic structure

  • We are missing a topological structure to measure length and distance

  • A norm on a vector space \(\calV\) over \(\bbR\) is a function \(\norm{\cdot}:\calV\to\bbR\) that satisfies:
    • Positive definiteness: \(\forall x\in\calV\) \(\norm{x}\geq 0\) with equality iff \(x=0\)
    • Homogeneity: \(\forall x\in\calV\) \(\forall\alpha\in\bbR\) \(\norm{\alpha x}=\abs{\alpha}\norm{x}\)
    • Subadditivity: \(\forall x,y\in\calV\) \(\norm{x+y}\leq \norm{x}+\norm{y}\)

  • \(\norm{x}\) measures a length, \(\norm{x-y}\) measures a distance

  • Image ? \(\bfx\in\bbR^d\qquad\norm[0]{\bfx}\eqdef\card{\set{i:x_i\neq 0}}\quad\norm[1]{\bfx}\eqdef\sum_{i=1}^d\abs{x_i}\quad \norm[2]{\bfx}\eqdef\sqrt{\sum_{i=1}^d x_i^2}\)

  • See board

Inner product

  • In addition to a topological and algebraic strucure, what if we want to do geometry?

  • An inner product space over \(\bbR\) is a vector space \(\calV\) equipped with a positive definite symmetric bilinear form \(\dotp{\cdot}{\cdot}:\calV\times\calV\to\bbR\) called an inner product

  • An inner product space is also called a pre-Hilbert space

  • See board

  • Unless stated otherwise, we will only deal with vector spaces on \(\bbR\) (things don’t change too much on \(\bbC\) but one should be a bit more careful) ## Induced norm and orthogonality

    • In an inner product space, an inner product induces a norm \(\norm{x} \eqdef \sqrt{\dotp{x}{x}}\)

    • A norm \(\norm{\cdot}\) is induced by an inner product on \(\calV\) iff \(\forall x,y\in\calV\) \(\norm{x}^2+\norm{y}^2 = \frac{1}{2}\left(\norm{x+y}^2+\norm{x-y}^2\right)\)

      If this is the case, the inner product is given by the polarization identity \[\dotp{x}{y}=\frac{1}{2}\left(\norm{x}^2+\norm{y}^2-\norm{x-y}^2\right)\]

  • An inner product satisfies \(\forall x,y\in\calV\) \(\dotp{x}{y}^2\leq\dotp{x}{x}\dotp{y}{y}\)

  • Two vectors \(x,y\in\calV\) are orthogonal if \(\dotp{x}{y}=0\). We write \(x\perp y\) for simplicity.

    A vector \(x\in\calV\) is orthogonal to a set \(\calS\subset\calV\) if \(\forall s\in\calS\) \(\dotp{x}{s}=0\). We write \(x\perp \calS\) for simplicity.
  • If \(x\perp y\) then \(\norm{x+y}^2=\norm{x}^2+\norm{y}^2\)