Projections

Skip to main content

\(\usepackage{amsmath,mathabx} \usepackage{mathpazo} \normalfont \def\bp{} \def\ep{} \def\R{{\mathbb R}} \def\Pos{{\mathbb P}} \def\Q{{\mathbb Q}} \def\Ir{{\mathbb I}} \def\N{{\mathbb N}} \def\Z{{\mathbb Z}} \def\T{{\mathbb T}} \def\H{{\mathbb H}} \def\mb{{\mathcal m}} \def\Mb{{\mathcal M}} \def\F{{\mathcal F}} \def\P{{\mathcal P}} \def\Qp{{\mathcal Q}} \def\a{{\alpha}} \def\d{{\delta}} \def\eps{{\epsilon}} \def\e{\varepsilon} \def\exp{\text{exp }} \def\g{{\gamma}} \def\G{{\Gamma}} \def\k{{\kappa}} \def\p{{\psi}} \def\t{{\tau}} \def\l{{\ell}} \def\la{{\lambda}} \def\La{{\Lambda}} \def\m{{\mu}} \def\n{{\nu}} \def\O{{\Omega}} \def\r{{\rho}} \def\th{{\theta}} \def\s{{\sigma}} \def\ze{{\zeta}} \def\ph{{\phi}} \def\Ph{{\Phi}} \def\Ps{{\Psi}} \def\U{{\boldsymbol{U}}} \def\A{{\boldsymbol{A}}} \def\B{{\boldsymbol{B}}} \def\C{{\boldsymbol{C}}} \def\D{{\boldsymbol{D}}} \def\E{{\boldsymbol{E}}} \def\J{{\boldsymbol{J}}} \def\I{{\boldsymbol{I}}} \def\L{{\boldsymbol{L}}} \def\Lam{{\boldsymbol{\Lambda}}} \def\M{{\boldsymbol{M}}} \def\P{{\boldsymbol{P}}} \def\X{{\boldsymbol{X}}} \def\lmatrix{\left(\begin{array}} \def\rmatrix{\end{array}\right)} \def\ldmatrix{\left|\begin{array}} \def\rdmatrix{\end{array}\right|} \def\bcm{} \def\pmkn{{(-1)}^{k+n}} \def\np{{}} \def\noi{{}} \def\lmatrix{\left(\begin{array}} \def\rmatrix{\end{array}\right)} \def\ldmatrix{\left|\begin{array}} \def\rdmatrix{\end{array}\right|} \def\ninN{{n{\in}\N}} \def\pcp2{\textcolor{forestgreen}\textbf{(CP)}} \def\pw2{\textcolor{magenta}\textbf{(CW)}} \newcommand{\chf}[1]{\chi_{{ }_{#1}}(x)} \def\ok{\textcolor{goodgreen}\textbf{(OK) }} \def\toinf{{\rightarrow \infty }} \def\tozero{{\rightarrow 0 }} \def\supp{{supp}} \def\Lsp{{\boldsymbol L}} \def\LtR{{\Lsp^2(\R)}} \def\Ra{{\,\,\Rightarrow\,\,}} \def\ra{{\,\rightarrow\,}} \def\Lfa{{\,\,\Leftarrow\,\,}} \def\LfRa{{\,\,\Leftrightarrow\,\,}} \def\lra{{\,\,\longrightarrow\,\,}} \def\Lra{{\,\,\Longrightarrow\,\,}} \def\LLa{{\,\,\Longleftarrow\,\,}} \def\LRa{{\,\,\Longleftrightarrow\,\,}} \def\fa{{\,\,\forall\,\,}} \def\ex{{\,\,\exists\,\,}} \def\st{{\,\,\ni\,\,}} \def\contr{{(\Rightarrow\,!\,\Leftarrow)}} \def\ntoinf{{\underset{n\rightarrow\infty}{\lim}\,}} \def\mtoinf{{\underset{m\rightarrow\infty}{\lim}\,\,}} \def\and{{\,\,\wedge\,\,}} \def\oor{{\,\,\vee\,\,}} \def\blt{$\bullet$} \def\tcb{\textcolor{blue}} \newcommand{\ve}[1]{\overrightarrow{\boldsymbol#1}} \newcommand{\vec}[1]{\overset{\rightharpoonup}{\boldsymbol#1}} \def\b{\vec{b}} \def\c{\vec{c}} \def\d{\vec{d}} \def\x{\vec{x}} \def\y{\vec{y}} \def\z{\vec{z}} \def\u{\vec{u}} \def\v{\vec{v}} \def\w{\vec{w}} def\z{\vec{z}} \newcommand{\ip}[2]{{\left\langle #1,#2\right\rangle}} \newcommand{\dpr}[2]{{#1\,{\boldsymbol \cdot}\,#2}} \newcommand{\row}[2]{{#1}_{#2\ast}} \newcommand{\col}[2]{{#1}_{\ast#2}} \newcommand{\wh}{\widehat} \newcommand{\wt}{\widetilde} \def\lmatrix{\left(\begin{array}} \def\rmatrix{\end{array}\right)} \def\0matrix{\mathcal{O}} \newcommand{\unit}[1]{\wh{\boldsymbol{#1}}} \newcommand{\lt}{<} \newcommand{\gt}{>} \newcommand{\amp}{&} \definecolor{fillinmathshade}{gray}{0.9} \newcommand{\fillinmath}[1]{\mathchoice{\colorbox{fillinmathshade}{$\displaystyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\textstyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\scriptstyle \phantom{\,#1\,}$}}{\colorbox{fillinmathshade}{$\scriptscriptstyle\phantom{\,#1\,}$}}} \)

Chapter 7 Projections

Consider a vector space \(V\) of dimension \(m,\) let \(S\) be a subspace of \(V,\) and let \(\v\in V.\) If \(\v\) is not in \(S\) we can ask - What is the closest vector in \(S\) to \(\v\) ? Formally, the answer is the orthogonal projection of \(\v\) to \(S.\) The notion of orthogonal projection models a variety of problems.

For example, if \(\A\x=\b\in\R^m\) has no solution - when \(\b\not\in\text{ col }(\A)\) - we may seek the closest \(\vec{b_\A}\in\text{ col }(\A)\) to \(\b\) and, possibly, an associated \(\x\in\R^n\) for which \(\A\x=\vec{b_A}.\) In this case we are solving the least-squares problem: given \(\A,\b\not\in\text{ col }(\A),\) find the vector \(\x\) at which \(\|\A\x-\b\|\) is minimized. This is called the arg min problem; that is, given \(\A,\b\not\in\text{ col }(\A),\) find

\begin{equation*} \x:=\text{ arg }\underset{\x\in\R^n}{\min}\left\|\A\x-\b\right\|. \end{equation*}

In the first section of this chapter we set up the problem and solve it, and in subsequent sections we consider a few useful applications of projections.