I regularly encounter the use of:
$$H X H^T$$
Where:
- $H$ is a $(n\times m)$ matrix, with $H^T$ it's transpose
- $X$ is a $(m \times m)$ matrix
If we rephrased this as a function (unquely defined for each H: $$f_H(X): \mathbb{R}^{m\times m} \to \mathbb{R}^{n \times n} : X \mapsto H X H^T$$
then what does the function $f_H$ do? Does it make sense to think of it that way?
Notionally what is this operation? I feel like there must be some intuitive notion of what it is doing.
It shows up (for example) in the Kalman Filter equations: $ \mathbf{P}_{k\mid k-1} = \mathbf{F}_{k} \mathbf{P}_{k-1\mid k-1} \mathbf{F}_{k}^{\text{T}} + \mathbf{Q}_{k}$.
Is seems like I am performing a operation which includes moving to some different space, and then performing the other half of the operation while moving back again.
Allow me to change the notation slightly by setting $A=X$ and $B=H^T$.
$B^TAB$ represents the change of basis of a quadratic form. If $B$ is invertible, then the matrices $A$ and $B^TAB$ are congruent.
More explicitly, if $A$ represents a quadratic form on $y\in\mathbb R^n$, $$f(y) = y^TAy,$$ but $y$ itself depends linearly on $x\in\mathbb R^m$, $$y = g(x) = Bx,$$ then the corresponding quadratic form on $x$ is represented by $B^TAB$: $$f(g(x)) = (Bx)^TA(Bx) = x^T(B^TAB)x.$$