7.2 Projection onto subspaces and Gram-Schmidt

What is a projection onto subspaces?

• Recall 1.5 Projection: we can write $\vec{x}={\vec{x}}_1+{\vec{x}}_2$ with ${\vec{x}}_1$ parallel to $\vec{v}\text{ and }{\vec{x}}_2$ perpendicular to $\vec{v}$:

  • 
    • ${\vec{x}}_1=pro{j}_{\vec{v}}(\vec{x})=\left(\frac{\vec{x}\cdot \vec{v}}{\mid |\vec{v}\mid {|}^2}\right)\vec{v}$
    • ${\vec{x}}_2=per{p}_{\vec{v}}(\vec{x})=\vec{x}-pro{j}_{\vec{v}}(\vec{x})$

  • In general:

    • if W is a subspace of ${\mathbb{R}}^n$ and $x\in {\mathbb{R}}^n$, we can write $\vec{x}={\vec{x}}_1+{\vec{x}}_2$, where ${\vec{x}}_1\in W$ and ${\vec{x}}_2$ is orthoganal to W

• Simply!

  • We know that $\vec{x}={\vec{x}}_1+{\vec{x}}_2$ and each is one another!

    • so we can just $pro{j}_w(\vec{x})=pro{j}_{\overrightarrow{v_1}}(\vec{x})+pro{j}_{{\vec{v}}_2}(\vec{x})$
      • 
      • and to get $per{p}_w(\vec{x})=\vec{x}-pro{j}_w(\vec{x})$

Orthogonal complement

• Let W be a subspace of ${\mathbb{R}}^n$.the orthogoanal complement of W, denoted $W^{\perp }$, is the set of all vectors that are orthogonal to every vector in W.

  • $W^{\perp }=(\vec{x}\in {\mathbb{R}}^n|\vec{x}\cdot \vec{w}=0\text{ for all }\vec{w}\in W)$

  • If $W=span({\vec{v}}_1,{\vec{v}}_2,\dots ,{\vec{v}}_k),W^{\perp }$can be found by writing ${\vec{v}}_i$ as rows of matrix A, and taking null(A)

Gram-Schmidt process

• If we are given basis $B=({\vec{v}}_1,\dots ,{\vec{v}}_k)$ for $span({\vec{v}}_1,\dots ,{\vec{v}}_k)$ which is not orthogonal, how do we transform it to an orthogonal basis for W?

  • The gram-Schmidt process constructs an orthogonal basis $(\vec{w},\dots ,{\vec{w}}_k)$ as follows

    • 1. ${\vec{w}}_1={\vec{v}}_1$
    • 2. let $W_1=span({\vec{w}}_1).$ set ${\vec{w}}_2=per{p}_{w_1}({\vec{v}}_2)$ (${\vec{w}}_2\text{ is orthognal to }{\vec{w}}_1$)
    • 3. let $W_2=span({\vec{w}}_1,{\vec{w}}_2).$ Set ${\vec{w}}_3=per{p}_{w_2}({\vec{v}}_3)$. (${\vec{w}}_3\text{ is orthogonal to }{\vec{w}}_1,{\vec{w}}_2$)
    • 4. proceed in this manner until all ${\vec{w}}_1,\dots ,{\vec{w}}_k$ are constructed then ${\vec{w}}_1,\dots ,{\vec{w}}_k$ is an orthogonal basis for W

      • notice that we are progressively taking perpendicular parts of vectors before adding them to the new set.