what is the intuition behind the dual hat matrix being the identity matrix?

115 Views Asked by At

Given some data matrix $\mathbf{X} \in \mathbb{R}^{N\times n}$ the $\textit{hat matrix}$ is $\mathbf{H} = \mathbf{X}(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T$ (henceforth called $\mathbf{H}_{Primal}(\mathbf{X})$). The hat matrix exists when the Gram matrix $\mathbf{X}^T\mathbf{X}$ is full rank (i.e. none perfect [multi-]colinearity), the $\textit{regularized hat matrix}$ $\mathbf{H}_{Reg. Primal}(\mathbf{X}) = \mathbf{X}(\mathbf{X}^T\mathbf{X} + \alpha\mathbf{I})^{-1}\mathbf{X}^T$ always exists for approriate $\alpha \in \mathbb{R}$.

The dualization of $\mathbf{H}$ is obtained by noting that $\mathbf{X}(\mathbf{X}^T\mathbf{X})^{-1} = (\mathbf{X}\mathbf{X}^T)^{-1}\mathbf{X}$ and in extension the dual hat matrix is $\mathbf{H}_{Dual}(\mathbf{X}) = (\mathbf{X}\mathbf{X}^T)^{-1}\mathbf{X}\mathbf{X}^T$ and the regularized dual hat matrix is $\mathbf{H}_{Reg. Dual}(\mathbf{X}) = (\mathbf{X}\mathbf{X}^T + \alpha\mathbf{I})^{-1}\mathbf{X}\mathbf{X}^T$.

Three relations strikes me as interesting and I'm looking for some intuition behind,

  1. It seems the dual hat matrix exists when it is equal to the identity matrix (i.e. $\mathbf{H}_{Dual}(\mathbf{X}) = \mathbf{I}$) why is this?
  2. When (if ever) does $\mathbf{H}_{Dual}(\mathbf{X})$ exists and $\mathbf{H}_{Primal}(\mathbf{X})$ does not (and vice versa)?
  3. Why is $\mathbf{H}_{Primal}(\mathbf{X}) = \mathbf{H}_{Dual}(\mathbf{X}^T)^T$?