I want to solve the independent component analysis (ICA) problem for a single time domain channel of input. The problem is, in my formulation, the mixing matrix has a special structure. I want to know if I can find a least-squares type projection of my estimated mixing matrix onto this special matrix structure.
Background:
Let $t_{n}$ be the $n$-th sample point, our received signal is then:
\begin{equation} x(t_{n}) = \sum_{m=1}^{M} a_{m}(t_{n}) \cdot s_{m}(t_{n}) = \mathbf{a}^{T}(t_{n}) \mathbf{s}(t_{n}) \end{equation} where $M$ is the number of sources, $\mathbf{a}(t_{n}) = [ a_{1}(t_{n}), \ldots, a_{M}(t_{n}) ]$ are the mixing coefficients, and $\mathbf{s}(t_{n}) = [s_{1}(t_{n}), \ldots, s_{M}(t_{n})]^{T}$ are the sources -- all at time $t_{n}$. Assuming our sampled signal is of length $L$, we can express this as a matrix-vector equation via: \begin{equation} \mathbf{x} = \mathbf{A} \mathbf{s} \end{equation} where $\mathbf{x} = [ x(t_{1}), x(t_{2}), \ldots, x(t_{L})]^{T}$, $\mathbf{s} = [ \mathbf{s}(t_{1}), \mathbf{s}(t_{2}), \ldots, \mathbf{s}(t_{L})]^{T}$ and: \begin{equation} \mathbf{A} = \begin{bmatrix} \mathbf{a}^{T}(t_{1}) & \mathbf{0}^{T} & \cdots & \mathbf{0}^{T} \\ \mathbf{0}^{T} & \mathbf{a}^{T}(t_{2}) & \cdots & \mathbf{0}^{T} \\ \vdots & \vdots & \ddots & \vdots \\ \mathbf{0}^{T} & \mathbf{0}^{T} & \cdots & \mathbf{a}^{T}(t_{L}) \end{bmatrix} \end{equation}
We see that this problem has been formulated as a standard ICA problem, and so, we could potentially solve it using ICA techniques. However -- standard ICA assumes, generally, that the matrix $\mathbf{A}$ is dense (or at least imposes no special structure on it).
Suppose that I apply some standard ICA technique and get a dense $\mathbf{\hat{A}}$. Is there a way I could project this matrix onto the space of matrices that have the form:
\begin{equation} \mathbf{A} = \begin{bmatrix} \mathbf{a}^{T}(t_{1}) & \mathbf{0}^{T} & \cdots & \mathbf{0}^{T} \\ \mathbf{0}^{T} & \mathbf{a}^{T}(t_{2}) & \cdots & \mathbf{0}^{T} \\ \vdots & \vdots & \ddots & \vdots \\ \mathbf{0}^{T} & \mathbf{0}^{T} & \cdots & \mathbf{a}^{T}(t_{L}) \end{bmatrix} \end{equation}
in such a way that the resulting projected matrix would be 'as close as possible' to the dense matrix in some way?
Yes. As it turns out, the result of finding the projection of $\hat A$ onto the vector space of matrices with the required zero pattern relative to the Frobenius inner product coincides with the result of the "common sense" approach: take any coefficients that are required to be zero and set them equal to zero.
Here's a quick proof: let $\mathbf e_1,\dots,\mathbf e_m$ denote the standard basis vectors of $\Bbb R^m$. Let $\mathbf E_{m,n}$ denote the matrix $\mathbf A$ corresponding to the mixing coefficients $\mathbf a(t)$ such that $\mathbf a(n) = \mathbf e_m$ and $\mathbf a(t) = \mathbf 0$ for all $t \neq n$. The set $\{E_{m,n} : 1 \leq m \leq M, \ 1 \leq n \leq L\}$ forms an orthonormal basis for the vector subspace of interest (i.e. the set of matrices of the desired form). As such, the projection of $\hat{\mathbf A}$ onto this subspace can be computed as $$ \mathbf A_{\text{proj}} = \sum_{m=1}^M \sum_{n = 1}^L \langle \hat{\mathbf A}, \mathbf E_{m,n} \rangle \mathbf E_{m,n}. $$ Verify that this is the matrix obtained by setting any entries of $\hat{\mathbf A}$ outside of the "block-diagonal" equal to zero.