prove that covariance matrix can be expressed as $XCX^\top$ with C is centering matrix

152 Views Asked by At

How to prove $\sum_{i=1}^{n}{(x_i-\bar{x}_n)(x_i-\bar{x}_n)^2}$ can be expressed as $XCX^\top$ with $C = I_n -\frac{1}{n} 11^\top $?

I read this answer but it did not lead me anywhere because I couldn't grasp the concept of the Kronecker product and also its beyond the syllabus.

1

There are 1 best solutions below

0
On

Expressions involving sums of indexed vectors can be very difficult to comprehend. My "goto" identity for unraveling them is the following $$\eqalign{ S &= \sum_{m=1}^M\sum_{n=1}^N A_{mn}\,y_m\,z_n^T \\ &= \sum_{m=1}^M\sum_{n=1}^N (Ye_m)A_{mn}(Ze_n)^T \\ &= Y\left(\sum_{m=1}^M\sum_{n=1}^N A_{mn}\;e_me_n^T\right)Z^T \\ &= YAZ^T }$$ where $y_m$ is the $m^{th}$ column of matrix $Y$, and $z_n$ is the $n^{th}$ column of matrix $Z$, and $A_{mn}$ are the elements of the matrix $A$.

In the current problem, $A$ is the identity matrix and $\,y=z=(x-\mu),\;$ where $\,\mu=\tfrac 1N(X{\tt1})$ is the row-mean vector of $X$.

For typing convenience, define the matrices $$\eqalign{ J &= \tfrac 1N {\tt1}{\tt1}^T \quad\implies\quad J^2=J=J^T \\ U &= \big[\mu\;\;\mu\;\;\ldots\;\mu \big] \;=\; \mu{\tt1}^T \;=\; XJ }$$ Then the sum can be expanded as $$\eqalign{ \sum_{n=1}^N (x_n-\mu)\,(x_n-\mu)^T &= \sum_{m=1}^M\sum_{n=1}^N \delta_{mn}\,(x_m-\mu)\,(x_n-\mu)^T \\ &= (X-U)I(X-U)^T \\ &= XX^T - UX^T - XU^T + UU^T \\ &= XX^T - (XJ)X^T - X(J^TX^T) + XJ^2X^T \\ &= XX^T - XJX^T \\ &= X(I-J)X^T \\ &= XCX^T \\ }$$