I am wondering if there is a method for finding a low-rank approximation to a block diagonal matrix such that the result is also block diagonal.
I have a problem where low-rank approximations might help. However, the structure of the resulting matrix cannot be compromised because of physical constraints.
In particular, I have a block diagonal matrix $D = diag(A,B,C)$, where $A$, $B$ and $C$ are dense non-identical matrices with no particular structure. The SVD of $D$ shows that a small fraction of the singular values contain most of the information in the matrix. Can I find a rank $k$ approximation, $D_{1}$ of $D$ such that $D_{1}$ is also block diagonal.
Even if it were known that this problem (or some variant of it for e.g., when $A$, $B$ and $C$ are symmetric) can be solved, or is NP hard, it would really help. I am a novice in optimization, so please forgive me if the question is rather trivial.