I have vectors $z, a \in \mathbb{R}^N$, and symmetric, positive-definite matrices $A, \Sigma \in \mathbb{R}^{N \times N}$. Both $A, \Sigma$ are covariance matrices.
I am given $z, a, A$ and want to solve for the unknown $\Sigma$.
I have a quadratic equation that looks like the following: $$ \Sigma - A^{T} \Sigma^{-1} A = (z - a)(z - a)^{T} $$ How should I solve this equation?
What follows gives a method for the approximation of the solution.
Let us call $v=z-a$, giving equation :
$$\Sigma - A \Sigma^{-1} A = vv^{T}\tag{1}$$
(I have dropped the "T" symbol on $A$).
Comparison of (1) with :
$$A - A A^{-1} A = 0\tag{2}$$
has made me think that we could look for a solution of the form :
$$\Sigma=A+E\tag{3}$$
with an "error term" $E$.
Plugging (3) into (1) :
$$A+E-A(A+E)^{-1}A=vv^T \ \ \iff \ \ A+E-A(A(I+A^{-1}E))^{-1}A=vv^T$$
$$A+E-A(I+\underbrace{A^{-1}E}_D)^{-1}A^{-1}A=vv^T \ \ \iff \ \ A+E-A(I-D+D^2+...)=vv^T$$
If, in the last parenthesis, we keep only the first 2 terms, we will obtain :
$$E+AD=vv^T \ \ \iff \ \ E+AA^{-1}E=vv^T \ \ \iff \ \ E=\frac12vv^T\tag{4}$$
Therefore, plugging (4) into (3), we will the following approximation :
$$\Sigma\approx A+\frac12vv^T$$
I have tested it on a computer : this appoximation is rather good under the condition that the series $I-D+D^2+...$ converges, i.e. that $A^{-1}$ is small enough...
We can get a better approximation by taking into account a third term in this series, giving :
$$\Sigma\approx A+\frac12vv^T+\frac18 vv^T A^{-1} vv^T$$
Using this formula, in a good part of cases, numerical experiments show a small discrepancy between the LHS and the RHS (around $10^{-3}$).
Besides, looking for an exact solution, I have explored different tracks. Two of them look interesting :
1) Let $u \in v^{\perp}$ (the hyperplane of vectors orthogonal to $v$).
Left multiplying (1) by $u^T$ and right multiplying it by $u$ gives $0$ on the RHS. Therefore
$$u^T\Sigma u = u^TA^{T} \Sigma^{-1}Au$$
i.e., the two quadratic forms coincide on this hyperplane.
2) Matrix
$$\begin{pmatrix}\Sigma&A\\A&\Sigma\end{pmatrix}$$
has the LHS of (1) for its "Schur complement".