Highly Nonlinear Approximations for Sparse Signal Representation


Construction of Oblique Projectors for signal splitting

Oblique projectors in the context of signal reconstruction were introduced in [2] and further analyzed in [3]. The application to signal splitting, also termed structured noise filtering, amongst a number of other applications, is discussed in [4]. For advanced theoretical studies of oblique projector operators in infinite dimensional spaces see [5,6]. We restrict our consideration to numerical constructions in finite dimension, with the aim of addressing the problem of signal splitting when the problem is ill posed.

Given $ {\cal{V}}$ and $ {\cal{W}^\bot}$ disjoint, i.e., such that $ {\cal{V}}\cap {\cal{W}^\bot}= \{0\},$ in order to provide a prescription for constructing $ \hat{E}_{{\cal{V}}{\cal{W}^\bot}}$ we proceed as follows. Firstly we define $ {\cal{S}}$ as the direct sum of $ {\cal{V}}$ and $ {\cal{W}^\bot}$, which we express as

$\displaystyle {\cal{S}}= {\cal{V}}\oplus {\cal{W}^\bot}.$

Let $ {\cal{W}}= ({\cal{W}^\bot})^\bot$ be the orthogonal complement of $ {\cal{W}^\bot}$ in $ {\cal{S}}$. Thus we have

$\displaystyle {\cal{S}}= {\cal{V}}\oplus {\cal{W}^\bot}={\cal{W}}\oplus^\bot {\cal{W}^\bot}.$

The operations $ \oplus$ and $ \oplus^\bot $ are termed direct and orthogonal sum, respectively.

Considering that $ \{v_i\}_{i=1}^M$ is a spanning set for $ {\cal{V}}$ a spanning set for $ {\cal{W}}$ is obtained as

$\displaystyle u_i=v_i - \hat{P}_{{\cal{W}^\bot}} v_i= \hat{P}_{{\cal{W}}} v_i,\; \;i=1,\ldots,M.$

Denoting as $ \{{\mathbf{{e}}}_i\}_{i=1}^M$ the standard orthonormal basis in $ \mathbb{F}^M$, i.e., the Euclidean inner product $ \langle {\mathbf{{e}}}_i, {\mathbf{{e}}}_j\rangle =\delta_{ij}$, we define the operators $ \hat{V}: \mathbb{F}^M \to {\cal{V}}$ and $ \hat{U}:\mathbb{F}^M \to {\cal{W}}$ as

$\displaystyle \hat{V}=\sum_{i=1}^M v_i \langle {\mathbf{{e}}}_i, \cdot \rangle ,
\hat{U}=\sum_{i=1}^M u_i \langle {\mathbf{{e}}}_i, \cdot \rangle .$

Thus the adjoint operators $ \hat{U}^\ast$ and $ \hat{V} ^\ast$ are

$\displaystyle \hat{V}^\ast=\sum_{i=1}^M {\mathbf{{e}}}_i\langle v_i , \cdot \ra...
\hat{U}^\ast=\sum_{i=1}^M {\mathbf{{e}}}_i\langle u_i , \cdot \rangle .$

It follows that $ \hat{P}_{{\cal{W}}} \hat{V}= \hat{U}$ and $ \hat{U}^\ast \hat{P}_{{\cal{W}}} = \hat{U}^\ast$ hence, $ \hat{G}: \mathbb{C}^M \to \mathbb{C}^M $ defined as:

$\displaystyle \hat{G}=\hat{U}^\ast \hat{V}= \hat{U}^\ast \hat{U}$

is self-adjoint and its matrix representation, $ G$, has elements

$\displaystyle g_{i,j}=\langle u_i,v_j\rangle = \langle \hat{P}_{{\cal{W}}} u_i,...
... \hat{P}_{{\cal{W}}} u_j\rangle = \langle u_i,u_j\rangle ,  \;\;i,j=1,\dots,M.$

From now on we restrict our signal space to be $ {\cal{S}}$, since we would like to build the oblique projector $ \hat{E}_{{\cal{V}}{\cal{W}^\bot}}$ onto $ {\cal{V}}$ and along $ {\cal{W}^\bot}$ having the form

$\displaystyle \hat{E}_{{\cal{V}}{\cal{W}^\bot}}=\sum_{i=1}^M v_i \langle w_i , \cdot \rangle .$ (3)

Clearly for the operator to map to zero every vector in $ {\cal{W}^\bot}$ the dual vectors $ \{w_i\}_{i=1}^M$ must span $ {\cal{W}}=({\cal{W}^\bot})^\bot={\mbox{\rm {span}}}\{u_i\}_{i=1}^M$. This entails that for each $ w_i$ there exists a set of coefficients $ \{b_{i,j}\}_{j=1}^M$ such that

$\displaystyle w_i= \sum_{i=1}^M b_{i,j} u_i,$ (4)

which guarantees that every $ w_i$ is orthogonal to all vectors in $ {\cal{W}^\bot}$ and therefore $ {\cal{W}^\bot}$ is included in the null space of $ \hat{E}_{{\cal{V}}{\cal{W}^\bot}}$. Moreover, since every signal, $ g$ say, in $ {\cal{S}}$ can be written as $ g=g_{{\cal{V}}} + g_{{\cal{W}^\bot}}$ with $ g_{{\cal{V}}} \in {{\cal{V}}}$ and $ g_{{\cal{W}^\bot}} \in {\cal{W}^\bot}$, the fact that $ \hat{E}_{{\cal{V}}{\cal{W}^\bot}}g = 0$ implies $ g_{{\cal{V}}}=0$. Hence, $ g=g_{{\cal{W}^\bot}}$, which implies that the null space of $ \hat{E}_{{\cal{V}}{\cal{W}^\bot}}$ restricted to $ {\cal{S}}$ is $ {\cal{W}^\bot}$.

In order for $ \hat{E}_{{\cal{V}}{\cal{W}^\bot}}$ to be a projector it is necessary that $ \hat{E}_{{\cal{V}}{\cal{W}^\bot}}^2=\hat{E}_{{\cal{V}}{\cal{W}^\bot}}$. As will be shown in the next proposition, if the coefficients $ b_{i,j}$ are the matrix elements of a generalised inverse of the matrix $ G$ this condition is fulfilled.

Proposition 1   If the coefficients $ b_{i,j}$ in (4) are the matrix elements of a generalised inverse of the matrix $ G$, which has elements $ g_{i,j}= \langle v_i,u_j\rangle ,  i,j=1,\ldots,M$, the operator in (3) is a projector.

Proof. For the measurement vectors in (4) to yield a projector of the form (3), the corresponding operator should be idempotent, i.e.,

$\displaystyle \sum_{n=1}^M \sum_{m=1}^M \sum_{i=1}^M\sum_{j=1}^M 
 v_i b_{i,j}\...
... =\sum_{i=1}^M\sum_{j=1}^M v_i \overline {b_{i,j}} \langle u_j , \cdot\rangle .$ (5)


$\displaystyle \hat{B} =\sum_{i=1}^M\sum_{j=1}^M {\mathbf{{e}}}_i\overline {b_{i,j}} \langle \mathbf{e}_j , \cdot \rangle$ (6)

and using the operators $ \hat{V}$ and $ \hat{U}^\ast$, as given above, the left hand side in (5) can be expressed as

$\displaystyle \hat{V} \hat{B}^\ast\hat{U}^\ast \hat{V} \hat{B}^\ast \hat{U}^\ast$ (7)

and the right hand side as

$\displaystyle \hat{V} \hat{B}^\ast\hat{U}^\ast.$ (8)

Assuming that $ \hat{B}^\ast$ is a generalised inverse of $ (\hat{U}^\ast \hat{V})$ indicated as $ \hat{B}^\ast = (\hat{U}\hat{V})^\dagger$ it satisfies (c.f. Section [*])

$\displaystyle \hat{B}^\ast(\hat{U}^\ast \hat{V})\hat{B}^\ast=\hat{B}^\ast,$ (9)

and therefore, from (7), the right hand side of (5) follows. Since $ \hat{B}^\ast = (\hat{U}^\ast\hat{V})^\dagger=
\hat{G}^\dagger$ and $ \hat{G}^\ast= \hat{G}$, we have $ \hat{B}= \hat{G}^\dagger$. Hence, if the elements $ b_{i,j}$ determining $ \hat{B}$ in (6) are the matrix elements of a generalised inverse of the matrix representation of $ \hat{G}$, the corresponding vectors $ \{w_i\}_{i=1}^n$ obtained by (4) yield an operator of the form (3), which is an oblique projector. $ \qedsymbol$

Property 1   Let $ \hat{E}_{{\cal{V}}{\cal{W}^\bot}}$ be the oblique projector onto $ {\cal{V}}$ and along $ {\cal{W}^\bot}$ and $ \hat{P}_{{\cal{W}}}$ the orthogonal projector onto $ {\cal{W}}= ({\cal{W}^\bot})^\bot$. Then the following relation holds

$\displaystyle \hat{P}_{{\cal{W}}} \hat{E}_{{\cal{V}}{\cal{W}^\bot}}= \hat{P}_{{\cal{W}}}.$

Proof. $ \hat{E}_{{\cal{V}}{\cal{W}^\bot}}$ given in (3) can be recast, in terms of operator $ \hat{V}$ and $ \hat{U}^\ast$, as:

$\displaystyle \hat{E}_{{\cal{V}}{\cal{W}^\bot}}= \hat{V} (\hat{U}^\ast \hat{V})^{\dagger} \hat{U}^\ast.$

Applying $ \hat{P}_{{\cal{W}}}$ both sides of the equation we obtain:

$\displaystyle \hat{P}_{{\cal{W}}} \hat{E}_{{\cal{V}}{\cal{W}^\bot}}= \hat{P}_{{...
...\dagger} \hat{U}^\ast=
\hat{U} (\hat{U}^\ast \hat{U})^{\dagger} \hat{U}^\ast,$

which is a well known form for the orthogonal projector onto $ {\cal{R}}(\hat{U})= {\cal{W}}$. $ \qedsymbol$

Remark 2   Notice that the operative steps for constructing an oblique projector are equivalent to those for constructing an orthogonal one. The difference being that in general the spaces $ {\mbox{\rm {span}}}\{v_i\}_{i=1}^M ={\cal{V}}\quad \text{and}\quad {\mbox{\rm {span}}}\{w_i\}_{i=1}^M= {\cal{W}}$ are different. For the special case $ u_i=v_i$, $ i=1,\ldots,M$, both sets of vectors span $ {\cal{V}}$ and we have an orthogonal projector onto $ {\cal{V}}$ along $ {\cal{V}}^\bot$.