Show that $Cov(AX) = A^T Cov(X) A$

Question

For simplicity, assume random variable $X$ has mean $0$. I want to prove that $$Cov(AX) = A^T Cov(X) A$$ where A is a scalar matrix.

I tried to prove it from the definition of covariance: $$Cov(AX) = E(AX)^T(AX) = E(X^TA^TAX)$$

I am wondering how do I bring $A$ out of the expectation. Thanks in advance.

Answer 1

Are you sure the statement you’re trying to prove isn’t $ACov(X)A^T$? Also, I believe in your definition of the covariance matrix you were applying the transpose operator to the wrong parentheses.

$$Cov(AX)=E[(AX)(AX)^T]=E(AXX^TA^T)=AE(XX^T)A^T=ACov(X)A^T$$

The statement you were trying to prove is actually incorrect. Let $x_1,x_2=\{-1,0,1\}$ be uniform random variables and $X=\begin{pmatrix} x_1 \\x_2\end{pmatrix}$ and $A= \begin{pmatrix}1 && 0 \\ 1 && 1 \end{pmatrix}$ then $$AX=\{ \begin{pmatrix} -1 \\ -2 \end{pmatrix}, \begin{pmatrix} -1 \\ -1 \end{pmatrix}, \begin{pmatrix} -1 \\ 0 \end{pmatrix}, \begin{pmatrix} 0 \\ -1 \end{pmatrix}, \begin{pmatrix} 0 \\ 0 \end{pmatrix}, \begin{pmatrix} 0 \\ 1 \end{pmatrix}, \begin{pmatrix} 1 \\ 0 \end{pmatrix}, \begin{pmatrix} 1 \\ 1 \end{pmatrix}, \begin{pmatrix} 1 \\ 2 \end{pmatrix} \}$$ so $$Cov(AX)=\begin{pmatrix}2/3 && 2/3 \\ 2/3 && 4/3 \end{pmatrix}$$ but $$A^TCov(X)A= \begin{pmatrix} 1 && 1 \\ 0 && 1 \end{pmatrix} \begin{pmatrix} 2/3 && 0 \\ 0 && 2/3 \end{pmatrix} \begin{pmatrix} 1 && 0 \\ 1 && 1 \end{pmatrix}= \begin{pmatrix}4/3 && 2/3 \\ 2/3 && 2/3 \end{pmatrix}$$

Answer 2

The covariance is defined as $$cov(AX) = E(AX - \mu_{AX}) (AX - \mu_{AX})^T$$ where in our particular case $$\mu_{AX} = E(AX) = AE(X) = 0 $$ This means that $$cov(AX) = E(AX) (AX )^T = E(AXX^TA^T) = AE(XX^T)A^T = Acov(X)A^T$$

Answer 3

I think what you’re trying to prove is $\text{Cov}(AX) = A \text{Cov}(X)A^T$. If $E(X) = 0$ then $\text{Cov}(X) = E[(X - E(X))(X-E(X))^T] = E(XX^T)$. Then $\text{Cov}(AX) = = E[AX (AX)^T] = E(AXX^TA^T) = A E(X X^T) A^T = A \text{Cov}(X)A^T$.