Under which condition is $\hat\Sigma\approx\frac{1}{T-1}(X'X)$

Question

Let $X$ be $T\times N$ random matrix. We are interested in the sample variance covariance matrix of $X$. It holds that \begin{align} \hat\Sigma&=\frac{1}{T-1}X'\left(I_T-\frac{1}{T}\iota_T\iota_T'\right)X\\ &=\frac{1}{T-1}\left(X'X-\frac{1}{T}\underbrace{X'\iota_T}_{=T\hat{\mu}}\,\,\underbrace{\iota_T'X}_{=T\hat{\mu}'}\right),\\ &=\frac{1}{T-1}\left(X'X-T\hat{\mu}\hat{\mu}'\right)\\ &=\frac{1}{T-1}X'X-\frac{T}{T-1}\hat{\mu}\hat{\mu}'\\ &\approx\frac{1}{T-1}X'X \end{align} where $I_T$ is the identity matrix of size $T$ and $\iota_T$ is a $T\times 1$ vector of $1$'s. I am looking for an argument or condition why the last approximation holds.

Answer 1

I am not sure if this statement hold for arbitrary choices of the matrix $X$. However, maybe it could be useful to analyse your question in a more statistical setting. I'll assume that the elements in the columns of your matrix $X$ follow an iid Normal distribution $$x_i\sim \text{iid} N(0,1)$$ You are interested into the magnitude of the matrix $Y:=X'X$ compared to $Z:=T\hat{\mu}\hat{\mu}'$. You know that the elements in the diagonal of $Z$ consist of $T\hat{\mu}^2$. For the equivalent diagonal elements of $Y$ you have: $$Y_{i,i}=\sum_{j=1}^{T}X_{i,j}^2>\max X_{i}^2$$ where $\max X_{i}^2$ is the maximum of the $i$-th column of $X$. The expected value for $\max X_{i}$ can be found in this post to be $$ \begin{eqnarray} m_T &=& \sqrt{\log \left(\frac{T^2}{2 \pi \log \left(\frac{T^2}{2\pi} \right)}\right)} \cdot \left(1 + \frac{\gamma}{\log (T)} + \mathcal{o} \left(\frac{1}{\log (T)} \right) \right) \end{eqnarray}$$ where $\gamma$ is the Euler-Mascheroni constant. For example, given $T=60$ this value is 2.421 and therefore $E(\max X_{i}^2)>5.861$ On the other hand side you can easily derive that $\hat{\mu_i}$ must be $\sqrt{m_T/T}\approx0.3125$ (Matlab approximates the Probability of $\hat{\mu}\geq0.31$ as $0$) in the setting described above to be equal to $\max X_i ^2$ - which is, by the way a very conservative estimator for $Y_{i,i}$. However, I have not figured out how to handle the off-diagonal elements yet.