Let $X$, matrix $n \times p$, with i.i.d. rows, mean $0$ and variance $\Sigma$. The usual unbiased estimator for $\Sigma$ is the sample covariance
$$ S = \frac{1}{n} X X^T. $$
Consider the case $p>n$. Here, $S$ is singular and makes difficult applications in which it is required to calculate its inverse (e.g., discriminants). Let's consider a regularized version:
$$ S^{*} = w_{1} l + w_{2} S. $$
We are interested in estimators of this form and with good properties. Consider the problem $$ \min _{w_{1}, w_{2}} E\left[\left\|S^{*}-\Sigma\right\|^{2}\right], $$ where $\|A\|=\sqrt{\operatorname{tr}\left(A A^T\right) / p}$ is the normalized Frobenius norm $A_{p \times p}$. Let
$$\mu = \langle\Sigma, \quad I \rangle, \quad \alpha^{2} = \|\Sigma-\mu I\|^{2},\quad \beta^{2}= E\left[\|S-\Sigma\|^{2}\right],\quad \delta^{2}= E\|S-\mu I\|^{2},$$
where $\left\langle A_{1}, A_{2}\right\rangle=\operatorname{tr}\left(A_{1} A_{2}^T\right) / p$. Show that $$\alpha^{2}+\beta^{2}=\delta^{2}$$
Attempt of solution
We have $\|\Sigma-\mu I\|^2$ is constant, so $E\left[\|\Sigma-\mu I\|^2\right]=\|\Sigma-\mu I\|^2 $. By linearity of expectation, $$\begin{equation}\label{eq1} E[b + a X]=b+ aE[X]. \end{equation}$$
Then, \begin{align*} \alpha^2 + \beta^2 & = \|\Sigma-\mu I\|^2+E\left[\|S-\Sigma\|^2\right]\\[1em] & = E\left[\|\Sigma-\mu I\|^2\right]+E\left[\|S-\Sigma\|^2\right]\\[1em] & = E\left[\|\Sigma-\mu I\|^2+\|S-\Sigma\|^2\right]\\[1em] & = E\left[\operatorname{tr}\left((\Sigma-\mu I)(\Sigma-\mu I)^T\right) / p+\operatorname{tr}\left((S-\Sigma)(S-\Sigma)^T\right) / p\right] , \end{align*} I'm a little bit confused, how can I go(continue) from there?
Here is the solution:
First notice $Cov(X,X)=E[S]=\Sigma$(*). Then,
$$ \delta^2=E\left[|| S-\mu I \|^{2}\right] =E\left[|| S-\Sigma+\Sigma-\mu I \|^{2}\right] \\[1em] =E\left[\|S-\Sigma\|^{2}\right]+E\left[|| \Sigma-\mu I \|^{2}\right]+2 E[\langle S-\Sigma, \Sigma-\mu I\rangle] \\[1em] \overset{(*)}{=}\|\Sigma-\mu I\|^{2}+E\left[|| S-\Sigma \|^{2}\right]+2\langle E[S-\Sigma], \Sigma-\mu I\rangle\\[1em] = \alpha^2 + \beta^2. $$