Solving a $3\times3$ non-linear system

41 Views Asked by At

Problem Formulation

Let $M\in\mathbb{R}^2$ be a covariance matrix given by the following factorization \begin{equation}\begin{aligned}M &=(RE)(RE)'\\ &=R\,EE'\, R' \end{aligned}\tag{1}\end{equation} where $'$ denote the transpose operator and $R$, $E$ are respectively the following rotation and extension matrices \begin{equation*}\tag{2}R\triangleq \left[\begin{array}{cc} c & -s \\ s & \phantom{-}c \end{array}\right] \qquad E\triangleq \left[\begin{array}{cc} a & 0 \\ 0 & b \end{array}\right]\end{equation*} where $\theta\in[-\pi,\pi]$, $a,b>0$ are given parameters and \begin{equation*} c\triangleq \cos \theta \qquad s\triangleq \sin \theta \end{equation*} I'm interested in the following inversion problem: given a covariance matrix \begin{equation*}\tag{3} M\triangleq \left[\begin{array}{cc} M_1 & M_{12} \\ M_{12} & M_2 \end{array}\right] \end{equation*} retrive the parameters $\theta,a, b$ that have generated the assigned $M$.

Naive attempt

By combining $(2)$ and $(3)$ in $(1)$ turns out the following non-linear system \begin{equation} \begin{aligned} M_1 &=c^2a^2+s^2b^2\\ M_2 &=s^2a^2+c^2b^2\\ M_{12} &=sc(a^2-b^2) \end{aligned}\tag{4} \end{equation} which, honestly speaking, I'm not able to resolve with respect to $\theta,a,b$. The only thing that cames immediately in my mind is the following \begin{equation*}M_1+M_2=a^2+b^2\end{equation*} but I don't know how to use it.

Refined attempt

Another approach that came in my mind is the following: by diagonalizing $M$ it turns out the following factorization \begin{equation*}\tag{5} M=D\,\Lambda\,D' \end{equation*} which resembles $(1)$. Here $D$ is orthonormal, which means that, for example, can be written as \begin{equation*}D\triangleq \left[\begin{array}{cc} c & -s \\ s & \phantom{-}c \end{array}\right]\end{equation*} where $\theta$ is a suitable rotation angle. On the other hand, here $\Lambda$ is diagonal and contains the eigenvalues $\lambda_1, \lambda_2$ of $M$, i.e. \begin{equation*}\Lambda\triangleq \left[\begin{array}{cc} \lambda_1 & 0 \\ 0 & \lambda_2 \end{array}\right]\end{equation*} The comparison of $(1)$ with $(5)$ gives the following relationships \begin{equation*}\begin{aligned} D &= R\\ \Lambda &= EE' \end{aligned}\end{equation*} In lieu of these two equations, holds the following:

  1. $\theta$ can be extracted from $D$, for example by computing \begin{equation*}\theta = \textrm{tan}^{-1}\left(\frac{D_{21}}{D_{11}}\right)=\textrm{tan}^{-1}\left(\frac{s}{c}\right)\end{equation*} where $D_{11}$, $D_{21}$ are the northwest, southwest entries of $D$;
  2. $a,b$ can be extracted from $\Lambda$ by computing \begin{equation*}a=\sqrt{\Lambda_{11}}=\sqrt{\lambda_1} \qquad b=\sqrt{\Lambda_{22}}=\sqrt{\lambda_2}\end{equation*} where $\Lambda_{11}$, $\Lambda_{22}$ are the diagonal entries of $\Lambda$.

Now the problem is reduced to find $s$, $c$, $\lambda_1$, $\lambda_2$, which amounts to just diagonalize $M$.


eigenvalues

By definition, $\lambda_1, \lambda_2$ are the two roots of the characteristic polynomial of $M$, which if $I_2$ is the $2\times2$ identity matrix, is given by \begin{equation*}\chi_M(\lambda)\triangleq\text{det}(\lambda I_2-M)=\lambda^2-(M_1+M_2)\lambda+M_1M_2-M_{12}^2\end{equation*} then, the two eigenvalues are \begin{equation*} \lambda_{1,2}=\frac{(M_1+M_2)\pm \sqrt{(M_1+M_2)^2-4(M_1M_2-M_{12}^2)}}{2} \end{equation*} consequently, the extension parameters $a,b$ are given by \begin{equation*} \boxed{ a=\sqrt{\lambda_1}=\sqrt{\frac{(M_1+M_2)+\sqrt{(M_1+M_2)^2-4(M_1M_2-M_{12}^2)}}{2}} } \end{equation*} \begin{equation*} \boxed{ b=\sqrt{\lambda_1}=\sqrt{\frac{(M_1+M_2)-\sqrt{(M_1+M_2)^2-4(M_1M_2-M_{12}^2)}}{2}} } \end{equation*}


eigenvectors

By definition, an eigenvector $v$ of $(M,\lambda)$ satisfies the equation \begin{equation*}Mv=\lambda v\end{equation*} and so, by writing component-wise $v=[\nu_1\,\,\nu_2]'$ it turns out the following linear system \begin{equation*}\begin{aligned} (M_1-\lambda)\, \nu_1 + M_{12}\, \nu_2 &=0\\ M_{12}\, \nu_1 + (M_{2}-\lambda)\, \nu_2 &=0\\ \end{aligned} \end{equation*} by fixing (for example) $\nu_2=1$ and by solving (for example) the first equation it turns out the following eigenvector \begin{equation*} v(\lambda)=\left[\begin{array}{cc} \frac{M_{12}}{\lambda-M_1} & 1 \end{array}\right]' \end{equation*} whose normalization is \begin{equation*}v_n(\lambda)\triangleq \frac{v(\lambda)}{\lVert v(\lambda)\rVert}=\frac{1}{\sqrt{\left(\frac{M_{12}}{\lambda-M_{1}}\right)^2+1}}\left[\begin{array}{cc} \frac{M_{12}}{\lambda-M_1} & 1 \end{array}\right]'\end{equation*} In conclusion,

  1. the searched eigenvectors are \begin{equation*} v_n^1 \triangleq v_n(\lambda_1)= \left[\begin{array}{c} \frac{M_{12}}{(\lambda_1-M_1)\sqrt{\left(\frac{M_{12}}{\lambda_1-M_1}\right)^2+1}} \\ \frac{1}{\sqrt{\left(\frac{M_{12}}{\lambda_1-M_1}\right)^2+1}} \\ \end{array}\right] \qquad v_n^2 \triangleq v_n(\lambda_2)= \left[\begin{array}{c} \frac{M_{12}}{(\lambda_2-M_1)\sqrt{\left(\frac{M_{12}}{\lambda_2-M_1}\right)^2+1}} \\ \frac{1}{\sqrt{\left(\frac{M_{12}}{\lambda_2-M_1}\right)^2+1}} \\ \end{array}\right]\\ \end{equation*}
  2. the diagonalizing matrix is \begin{equation*}D=\left[\begin{array}{cc} v_n^1 & v_n^2 \end{array}\right] = \left[\begin{array}{cc} \frac{M_{12}}{(\lambda_1-M_1)\sqrt{\left(\frac{M_{12}}{\lambda_1-M_1}\right)^2+1}} & \frac{M_{12}}{(\lambda_2-M_1)\sqrt{\left(\frac{M_{12}}{\lambda_2-M_1}\right)^2+1}} \\ \frac{1}{\sqrt{\left(\frac{M_{12}}{\lambda_1-M_1}\right)^2+1}} & \frac{1}{\sqrt{\left(\frac{M_{12}}{\lambda_2-M_1}\right)^2+1}} \end{array}\right] \end{equation*}
  3. the trigonometric parameters are \begin{equation*}\begin{aligned} s&=D_{21} = \frac{1}{\sqrt{\left(\frac{M_{12}}{\lambda_1-M_1}\right)^2+1}} \\ c&=D_{11} = \frac{M_{12}}{(\lambda_1-M_1)\sqrt{\left(\frac{M_{12}}{\lambda_1-M_1}\right)^2+1}} \\ \end{aligned} \end{equation*}
  4. the rotation angle is \begin{equation*} \boxed{ \theta = \textrm{tan}^{-1}\left(\frac{D_{21}}{D_{11}}\right)=\textrm{tan}^{-1}\left(\frac{\lambda_1-M_1}{M_{12}}\right) }\end{equation*}

Sanity checks

My solution is pretty much involved, and I don't consider it reliable at all. In order to validate my results I want to make some sanity checks.

  1. If my solution is correct, plugging $(4)$ inside the final expressions of $a,b,\theta$ must give an identity. The problem is that this operations are not straightforwad.
  2. The diagonalizing matrix $D$ is redundant, in the sense that there are multiple ways to compute the trigonometric parameters. Here the sanity checks are the following: is it true that $D_{11}=D_{22}$, $D_{21}=-D_{12}$? Once again, here the computations are not straightforward.

Observation

For what concern the sanity check about the proposed solutions for $a,b$, it should be noted that, since $M_1+M_2=a^2+b^2$, if everything is ok it must be \begin{equation*} \sqrt{(M_1+M_2)^2-4(M_1M_2-M_{12}^2)}=a^2-b^2 \end{equation*} or, equivalently, \begin{equation*} M_1M_2-M_{12}^2 = a^2b^2 \end{equation*} According to my solution, is this relation true?

Questions

  1. Is there any chance to solve $(4)$ with a simpler strategy than the one proposed by me?
  2. How can I verify the sanity checks?
  3. Does it make sense the idea to diagonalize $M$?