I am calculating the eigen vectors from a set of M grayscale faces images and I am using two methods. I would expect to get the same results but calculation gave me the different outcomes.
Suppose I have been given the following 2 images, each has only 10 pixels. $ M= \left[ { \begin{array}{cc} 2.5 & 2.4\\ 0.5 & 0.7\\ 2.2 & 2.9\\ 1.9 & 2.2\\ 3.1 & 3.0\\ 2.3 & 2.7\\ 2.0 & 1.6\\ 1.0 & 1.1\\ 1.5 & 1.6\\ 1.1 & 0.9 \end{array} } \right] $
Subtracted Mean from $M$ to get Matrix $ \bar{M}= \left[ { \begin{array}{cc} 0.6900 & 0.4900 \\ -1.3100 & -1.2100 \\ 0.3900 & 0.9900 \\ 0.0900 & 0.2900 \\ 1.2900 & 1.0900 \\ 0.4900 & 0.7900 \\ 0.1900 & -0.3100 \\ -0.8100 & -0.8100 \\ -0.3100 & -0.3100 \\ -0.7100 & -1.0100 \end{array} } \right] $
Obtain a covariance matrix \
\begin{eqnarray} C = \bar{M}\bar{M}^{T}= \\ \left[ { \begin{array}{cccccccccc} 0.0796 & -0.1663 & 0.0838 & 0.0227 & 0.1582 & 0.0806 & -0.0023 & -0.1062 & -0.0406 & -0.1094 \\ -0.1663 & 0.3534 & -0.1899 & -0.0521 & -0.3343 & -0.1775 & 0.0140 & 0.2268 & 0.0868 & 0.2391 \\ 0.0838 & -0.1899 & 0.1258 & 0.0358 & 0.1758 & 0.1081 & -0.0259 & -0.1242 & -0.0475 & -0.1419 \\ 0.0227 & -0.0521 & 0.0358 & 0.0102 & 0.0480 & 0.0304 & -0.0081 & -0.0342 & -0.0131 & -0.0396 \\ 0.1582 & -0.3343 & 0.1758 & 0.0480 & 0.3169 & 0.1659 & -0.0103 & -0.2142 & -0.0820 & -0.2241 \\ 0.0806 & -0.1775 & 0.1081 & 0.0304 & 0.1659 & 0.0960 & -0.0169 & -0.1152 & -0.0441 & -0.1273 \\ -0.0023 & 0.0140 & -0.0259 & -0.0081 & -0.0103 & -0.0169 & 0.0147 & 0.0108 & 0.0041 & 0.0198 \\ -0.1062 & 0.2268 & -0.1242 & -0.0342 & -0.2142 & -0.1152 & 0.0108 & 0.1458 & 0.0558 & 0.1548 \\ -0.0406 & 0.0868 & -0.0475 & -0.0131 & -0.0820 & -0.0441 & 0.0041 & 0.0558 & 0.0214 & 0.0592 \\ -0.1094 & 0.2391 & -0.1419 & -0.0396 & -0.2241 & -0.1273 & 0.0198 &0.1548 & 0.0592 & 0.1694 \end{array} } \right] \end{eqnarray}
The first method to calculate the eigenvectors of the covariance matrix $C$ direct SVD on C
\begin{eqnarray} E = \\ \left[ { \begin{array}{cccccccccc} 0.2436 & -0.2635 & 0.6008 & 0.0000 & 0.0000 & 0.0000 & 0.0000 & 0.0000 & 0.0000 & 0.7143 \\ -0.5229 & 0.2149 & -0.4587 & -0.1691 & 0.1472 & -0.0184 & 0.0423 & 0.0033 & 0.0603 & 0.6434 \\ 0.2919 & 0.5783 & 0.0912 & 0.2501 & 0.3541 & 0.0279 & -0.0012 & -0.0504 & 0.6160 & 0.0371 \\ 0.0807 & 0.1962 & 0.0941 & -0.3549 & 0.3479 & -0.1255 & -0.6593 & -0.3834 & -0.3172 & -0.0343 \\ 0.4930 & -0.3152 & -0.5340 & 0.2090 & 0.1815 & 0.0743 & -0.3509 & 0.3728 & -0.0313 & 0.1648 \\ 0.2686 & 0.2637 & -0.1105 & 0.0079 & -0.2758 & -0.8420 & 0.1352 & 0.0770 & -0.1643 & 0.0987 \\ -0.0292 & -0.5263 & -0.1206 & -0.3010 & 0.0275 & -0.3391 & -0.0506 & -0.3147 & 0.6268 & -0.0828 \\ -0.3367 & 0.0698 & 0.2719 & -0.0962 & -0.1719 & -0.1453 & -0.4934 & 0.6566 & 0.2562 & -0.0882 \\ -0.1289 & 0.0267 & -0.0803 & 0.5531 & -0.5421 & 0.0516 & -0.4190 & -0.4188 & 0.0902 & 0.1213 \\ -0.3600 & -0.2448 & 0.1382 & 0.5774 & 0.5465 & -0.3604 & 0.0257 & -0.0023 & -0.1475 & -0.0838 \end{array} } \right] \end{eqnarray}
The eigen values are \begin{eqnarray} \Lambda = \\ \left[ { \begin{array}{cccccccccc} 1.2840 & 0.0491 & 0.0000 & 0.0000 & 0.0000 & 0.0000 & 0.0000 & 0.0000 & 0.0000 & 0.0000 \end{array} } \right] \end{eqnarray}
The second method reduces compleixty by calculating the inner product matrix $L=\bar{M}^{T}\bar{M}$ of the size 2×2. Then find it's eigenvectors $v_i$,i=1,2 of L (of the 2nd dimension).
\begin{eqnarray} L = \left[ { \begin{array}{cc} 0.6166 & 0.6154 \\ 0.6154 & 0.7166 \end{array} } \right] \end{eqnarray}
\begin{eqnarray} V = \left[ { \begin{array}{cc} -0.6779 & 0.7352 \\ -0.7352 & -0.6779 \end{array} } \right] \end{eqnarray}
\begin{eqnarray} \lambda = \left[ { \begin{array}{cc} 1.2840 & 0.0491 \end{array} } \right] \end{eqnarray}
Note that if $Lv_{i} = \lambda_{i}$, then
\begin{eqnarray} \bar{M}Lv_{i} = \lambda_{i}\bar{M}v_{i} \\ \bar{M}\bar{M}^{T}v^{i} = \lambda_{i}\bar{M}v_{i} \\ C\bar{M}v_{i} = \lambda_{i}\bar{M}v_{i} \end{eqnarray}
and hence $u_i=\bar{M}v_i$ and $\lambda_{i}$ are respectively the 2 eigenvectors (of 10-th dimension) and eigenvalues of C. After normalizing $u_i$, such that $|u_i|=1$
\begin{eqnarray} U = \left[ { \begin{array}{cc} -0.2436 & 0.2635 \\ 0.5229 & -0.2149 \\ -0.2919 & -0.5783 \\ -0.0807 & -0.1962 \\ -0.4930 & 0.3152 \\ -0.2686 & -0.2637 \\ 0.0292 & 0.5263 \\ 0.3367 & -0.0698 \\ 0.1289 & -0.0267 \\ 0.3600 & 0.2448 \end{array} } \right] \end{eqnarray}
The two eigen vector $U$ calculated in the second method is different to the two dominant eigen vectors calcualted in the first method by multiplying (-1).
Could someone explain to me what causes the difference? Is my calculation wrong or the result meant to be different by sign?
Many Thanks in advance.