How covariance matrix represent data?

Question

I know that covariance matrix represent data as eigenvector of it but I want to understand how could covariance matrix represent data mathematically?

How could those values help for representing some kind of average of data?

Multiplying any random vector repeatedly with covariance matrix results in the eigenvector (some scale of it) but how? (I try for a 2d vector space with x and y values and get some equations but I could not prove it.) Could we prove that the covariance matrix represent the data geometrically or in mathematical sense?

Answer 1

Actually I find a comprehensive answer in the following article with proofs that covariance matrix is a kind of transformation of white data which contains any random vectors. I am convinced by its arguments with matrix equations and graphical explanations.

The covariance matrix defines the shape of the data. Diagonal spread is captured by the covariance, while axis-aligned spread is captured by the variance. The covariance matrix represents a linear transformation of the original data. In this article we showed that the covariance matrix of observed data is directly related to a linear transformation of white, uncorrelated data. This linear transformation is completely defined by the eigenvectors and eigenvalues of the data. While the eigenvectors represent the rotation matrix, the eigenvalues correspond to the square of the scaling factor in each dimension.

https://www.visiondummy.com/2014/04/geometric-interpretation-covariance-matrix/

In case link breaks https://web.archive.org/web/20200811123439/https://www.visiondummy.com/2014/04/geometric-interpretation-covariance-matrix/