Finding distribution of difference of Bayesian and single point linear regression

52 Views Asked by At

In task the theorem needed to be proven is :

If $y = X\beta + w$ with $\beta \sim N(0, t \cdot I_d)$ and $w \sim N(0, I_n)$ then $$E\frac{1}{n}\|X\hat{\beta}_{mean} - X\beta \| = \frac{1}{n}\sum_{i = 1}^n \frac{\lambda_i}{\lambda_i + \frac{1}{t}} $$

where $\lambda_1, ... , \lambda_d \ge 0$ are the eigenvalues of $X^TX$ and $\hat{\beta}_{mean} = E[{\beta} | y]$ (posterior parameter estimation). $\beta = Ay + z$ where $A \in \mathbb{R}^{d\times n}$ and $y$ and $z$ are independent Gaussian distribution and $\beta$ from there is Gaussian as well.

Proof for this goes from some previous corollary which states that $\hat{\beta}_{mean}(y) = B_yy$ and $X\beta - B_ty \sim N(0, B_t)$, where $B_t$ is matrix defined from previous corollary as we said and equals $B_t = X(X^TX + \frac{1}{t}I_d)^{-1}X^T$.

I don't understand how they came up to $\hat{\beta}_{mean}(y) = B_yy$ and covariance in $X\beta - B_ty \sim N(0, B_t)$?