Invertibility of the symmteric matrix observed under least squaress solution

21 Views Asked by At

I am having trouble in proving the following statement

Let $X$ be a $K \times d$ matrix with rank $d$ and let $W$ be a diagonal matrix with $d$ non-zero positive elements on the diagonal, that is rank of $W$ is $d$ as well. Show that $X^\top W X$ is invertible. Assume we are working in the field of real numbers.

I am able to decompose $X$ in a SVD fashion of $X = U \Sigma V^\top$ to get \begin{align*} \mathrm{det}(X^\top W X)& = \mathrm{det}(V\Sigma^\top U^\top W U\Sigma V) \\ & = \mathrm{det}(\Sigma^\top U^\top W U \Sigma) \end{align*} But I am getting stuck after this. For $K=3$ and $d=2$, assuming first 2 elements of $W$ on the diagonal are non-zer0, i was able to show that the above quantity boils down to \begin{align*} \mathrm{det}(\Sigma^\top U^\top W U \Sigma) = \sigma_1^2\sigma_2^2(\lambda_1u_{11}^2 + \lambda_2 u_{12}^2)(\lambda_1u_{21}^2 +\lambda_2u_{22}^2) > 0, \end{align*} where $\lambda_i$ are the alements of the $W$ matrix. I would assume the statement to be true. Any help is appreciated as always. Thanks