Prove that the values of the random vector $\vec{Y}$ are uncorrelated

76 Views Asked by At

Let $\vec{X}^{T}=\begin{bmatrix}X_{1} & X_{2} & \ldots & X_{n}\end{bmatrix}$ be a random vector. Let's define: $$ \vec{Y}=A\left(\vec{X}-E(\vec{X})\right) $$ to be the Karhunen–Loève transform. I was asked to prove that the values of the vector $\vec{Y}$ are uncorrelated. Could not find a definition for correlation of random-vectors/matrices (only for random variables) but my solution is based on a similar solution I saw.

What I did: $$ \begin{align*} E\left(\vec{Y}\vec{Y}^{T}\right)&\overset{(1)}{=}E\left[\left(A\left(\vec{X}-E\left(\vec{X}\right)\right)\right)\cdot\left(A\left(\vec{X}-E\left(\vec{X}\right)\right)\right)^{T}\right]\\&\overset{(2)}{=}E\left[\left(A\left(\vec{X}-E\left(\vec{X}\right)\right)\right)\cdot\left(\left(\vec{X}-E\left(\vec{X}\right)\right)^{T}A^{T}\right)\right]\\&\overset{(3)}{=}AE\left[\left(\vec{X}-E\left(\vec{X}\right)\right)\cdot\left(\vec{X}-E\left(\vec{X}\right)\right)^{T}\right]A^{T}\\&\overset{(4)}{=}AC_{X}A^{T}\overset{(5)}{=}A\left(A^{T}DA\right)A^{T}\overset{(6)}{=}IDI\overset{(7)}{=}D \end{align*} $$

where:

  1. Because $\vec{Y}=A\left(\vec{X}-E\left(\vec{X}\right)\right)$.
  2. Because $\left(AB\right)^{T}=B^{T}A^{T}$.
  3. Linear of the epexcted value.
  4. Because $C_X=E\left[\left(\vec{X}-E\left(\vec{X}\right)\right)\cdot\left(\vec{X}-E\left(\vec{X}\right)\right)^{T}\right]$ (Covariance matrix definition).
  5. Due to the Spectral theorem: $C_{X}=A^{T}DA$.
  6. Because $AA^{T}=A^{T}A=I$.
  7. Because $IA=AI=A$.

So we get the expected value of $\vec{Y}\vec{Y}^{T}$ is the diagonal matrix $D$ which is constant. This means that elements in the $\vec{Y}$ vector are uncorrelated.

Now, a few things. First of all, I cheated. Due to the Spectral theorem I actually get $C_X=ADA^T$ and not $C_X=A^TDA$ but then I can't continue. Secondly, why if $E(\vec{Y}\vec{Y}^T)$ is constant then it means that the elements of $\vec{Y}$ are uncorrelated? As I mentioned, I could not find the definition.

My proof is based on a similar proof where $\vec{Y}=Q^{T}\left(\vec{X}-E\left(\vec{X}\right)\right)$ and in that case I get $E\left(\vec{Y}\vec{Y}^{T}\right)=D$ because $C_X=QDQ^{T}$.

EDIT: As I understand the elements in vector $\vec{X}^{T}=\begin{bmatrix}X_{1} & X_{2} & \ldots & X_{n}\end{bmatrix}$ are uncorrelated if $Cov\left(X_{i},X_{j}\right)=0$ for every $i\neq j$. So as I understand, I need to calculate $Cov\left(Y_{i},Y_{j}\right)=0$ for $i\neq j$. Why did they calculate $E(YY^T)$?

EDIT2: Looking on page 6 of this booklet, I see:

Given the covariance matrix $C_X$ the KLT transformation matrix $A$ is defined such that after transformation of each vector via: $$y = A(x − x_m)$$ the covariance matrix of the vectors $y$ which we denote $C_y$ is diagonal - so that each pixel of the output vector set is uncorrelated with every other pixel.

I guess this is what I'm trying to prove.