Highly Nonlinear Approximations for Sparse Signal Representation


Linear operators and linear functionals

Let $ {\cal{V}}_1$ and $ {\cal{V}}_2$ be vectors spaces. A mapping $ \hat{A}: {\cal{V}}_1 \to {\cal{V}}_2$ is a linear operator if

$\displaystyle \hat{A}(v + w) = \hat{A}v + \hat{A}w,\quad \hat{A}(av)= a \hat{A}v,$

for all $ v, w \in {\cal{V}}_1$ and $ a\in F$. $ {\cal{V}}_1$ is called the domain of $ \hat{A}$ and $ {\cal{V}}_2$ its codomain or image. If the codomain of a linear operator is a scalar field, the operator is called a linear functional on $ {\cal{V}}_1$. The set of all linear functionals on $ {\cal{V}}_1$ is called the dual space of $ {\cal{V}}_1$.

The adjoint of an operator $ \hat{A}: {\cal{V}}_1 \to {\cal{V}}_2$ is the unique operator $ \hat{A}^\ast$ satisfying that

$\displaystyle \langle \hat{A} g_1, g_2\rangle = \langle g_1, \hat{A}^\ast g_2\rangle .$

If $ \hat{A}^\ast= \hat{A}$ the operator is self-adjoint or Hermitian

An operator $ \hat{A}: {\cal{V}}_1 \to {\cal{V}}_2$ has an inverse if there exists $ \hat{A}^{-1} : {\cal{V}}_2 \to
{\cal{V}}_1$ such that

$\displaystyle \hat{A}^{-1} \hat A =\hat{I}_{{\cal{V}}_1}$   and$\displaystyle \quad
\hat A \hat{A}^{-1} =\hat{I}_{{\cal{V}}_2},$

where $ \hat{I}_{{\cal{V}}_1}$ and $ \hat{I}_{{\cal{V}}_2}$ denote the identity operators in $ {{\cal{V}}_1}$ and $ {{\cal{V}}_2}$, respectively. By a generalised inverse we shall mean an operator $ \hat{A}^{\dagger}$ satisfying the following conditions
$\displaystyle \hat{A} \hat{A}^{\dagger} \hat{A}$ $\displaystyle =$ $\displaystyle \hat{A}$  
$\displaystyle \hat{A}^{\dagger} \hat{A} \hat{A}^{\dagger}$ $\displaystyle =$ $\displaystyle \hat{A}^{\dagger}.$  

The unique generalized inverse satisfying
$\displaystyle (\hat{A} \hat{A}^{\dagger})^\ast$ $\displaystyle =$ $\displaystyle \hat{A} \hat{A}^{\dagger}$  
$\displaystyle (\hat{A}^{\dagger} \hat{A})^\ast$ $\displaystyle =$ $\displaystyle \hat{A}^{\dagger} \hat{A}.$  

is known as the Moore Penrose pseudoinverse.