Suppose I have two $N \times M$, $N \gg M$ matrices $\mathbf{A}$ and $\mathbf{B}$. Both have a structure and are generated using an analytical description from a set of iid random variables $X_i$. Both are assumed to have full column rank. For example:
$$ a_{ij} = X_i^j \\ b_{ij} = \sum_{r=1}^R X_{(i-1)R+r}^j $$
$a_{ij}$ and $b_{ij}$ are the elements of $\mathbf{A}$ and $\mathbf{B}$ and $X_i$ random variables, e.g. Gaussian iid. Both are used in a LS setting, for example:
$$ \mathbf{y} = \mathbf{A}\mathbf{c} + \mathbf{e} \\ \mathbf{\hat c} = (\mathbf{A}^H \mathbf{A})^{-1} \mathbf{A}^H \mathbf{y} $$
I would like to find a proxy for the MSE $\|\mathbf{\hat c}-\mathbf{c}\|_2$ whose expected value I can evaluate analytically. With proxy I mean that the quantity is consistently smaller (or greater) if the MSE is smaller (or the other way round). My objective is to proof that a matrix generated with the second method is always "better" than a matrix generated with the other method.
I was thinking about the following proxies:
- Condition number
- "A optimality": $\operatorname{Tr}(\mathbf{A}^H \mathbf{A})^{-1}$ (this is proportional to MSE!)
- "D optimality": $\operatorname{det}(\mathbf{A}^H \mathbf{A})$ or $\operatorname{det}(\mathbf{A}^H \mathbf{A})^{-1}$
- "E optimality": $\operatorname{min}\operatorname{max}\operatorname{eig}(\mathbf{A}^H \mathbf{A})^{-1}$ or $\operatorname{max}\operatorname{min}\operatorname{eig}(\mathbf{A}^H \mathbf{A})$
- "T optimality": $\operatorname{Trace}(\mathbf{A}^H \mathbf{A})$
- Frame Potential: $\operatorname{Trace}((\mathbf{A}^H \mathbf{A})^H \mathbf{A}^H \mathbf{A})$
1) and 4) requires eigen-/singularvalue calculation which can't be done analytically. 2) requires matrix inversion which can't be done analytically. 3) requires determinant which can't be done analytically. I can easily calculate 5. analytically but it does not always seem to be consistent with the MSE. I can also analytically calculate 6. but this is only consistent if $\mathbf{A}$ and $\mathbf{B}$ have unit-norm rows which is not the case for my setup.
Basically I have an analytical expression for each entry (i.e., $a_{nm}$ and $b_{nm}$). However, I cannot do eigenvalues, matrix inversion etc.