If I have a covariance matrix $\Sigma$, and $x^T \Sigma^{-1} x = ||Ax||_2^2$. What exactly is the significance of $A$?
I tried solving for $A$ with an eigenvalue decomposition of $\Sigma$ as follows,
$x^T \Sigma^{-1} x = x^T (Q \Lambda Q^T)^{-1}x = x^T Q\Lambda^{-1}Q^{-1} x = x^TA^TAx$ and this is where I get stuck. I am unable to figure out a closed form solution to $A$.
Assuming that the entries of the matrix are real,
$||Ax||_2^2 =<Ax,Ax>$
$ =<x,A^*Ax>$
$=<x,A^TAx>$
$=x^T\Sigma^{-1}x$
$=<x,\Sigma^{-1}x>$
Since I don't have the proof , let me assume that $\Sigma^{-1}$ is positive definite.
Then we have $\Sigma^{-1}=A^*A$ and therefore $A$ is the unique nonnegative square root of $\Sigma^{-1}$.
The algorithm to compute it are given here : http://en.wikipedia.org/wiki/Square_root_of_a_matrix#Square_roots_of_positive_operators