How to Change Summation Expression $\sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i$ into Matrix Expression

Question

Let $\mathbf{X}_i$ be a $G \times K$ matrix, and suppose are $i=1,...,N$ of these matrices. Note that \begin{align} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{X}_i &= \begin{bmatrix} \mathbf{X}_1^{\top} & \mathbf{X}_2^{\top} & \cdots & \mathbf{X}_N^{\top} \end{bmatrix} \begin{bmatrix} \mathbf{X}_1 \\ \mathbf{X}_2 \\ \vdots \\ \mathbf{X}_N \end{bmatrix} \\ &=\begin{bmatrix} \mathbf{X}_1 \\ \mathbf{X}_2 \\ \vdots \\ \mathbf{X}_N \end{bmatrix}^{\top} \begin{bmatrix} \mathbf{X}_1 \\ \mathbf{X}_2 \\ \vdots \\ \mathbf{X}_N \end{bmatrix} \\ &=\mathbf{X}_{\bullet}^{\top}\mathbf{X}_{\bullet} \end{align} where $\mathbf{X}_{\bullet}$ is an $NG \times K$ matrix whose $i^{th}$ row is $\mathbf{X}_i$.

I'd like to do something similar with this summation: \begin{equation} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i \end{equation} where $\mathbf{\Omega}$ is some arbitrary symmetric $G \times G$ matrix, say for illustration \begin{equation} \mathbf{\Omega} =\begin{bmatrix} g_{11} & g_{12} & \cdots & g_{1G} \\ g_{12} & g_{22} & \cdots & g_{2G} \\ \vdots & \vdots & \ddots & \vdots \\ g_{1G} & g_{2G} & \cdots & g_{GG} \end{bmatrix} \end{equation}

This problem is not so simple as the first one, because now I have some $\mathbf{\Omega}^{-1}$ matrix in the middle of the sum. However, I think it can still be expressed in matrix notation.

For those curious, this matrix algebra is behind the theory of Generalized Least Squares. Thanks very much for your help!

Edit I think I have figured it out. I have posted an answer below -- please let me know if you think it's correct!

Answer 1

Let's try this. \begin{align} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i&=\mathbf{X}_1\mathbf{\Omega}^{-1}\mathbf{X}_1+\mathbf{X}_2\mathbf{\Omega}^{-1}\mathbf{X}_2+\cdots+\mathbf{X}_N\mathbf{\Omega}^{-1}\mathbf{X}_N \\ &=\begin{bmatrix} \mathbf{X}_1^{\top} & \mathbf{X}_2^{\top} & \cdots & \mathbf{X}_N^{\top} \end{bmatrix} \begin{bmatrix} \mathbf{\Omega}^{-1}\mathbf{X}_1 \\ \mathbf{\Omega}^{-1}\mathbf{X}_2 \\ \vdots \\ \mathbf{\Omega}^{-1}\mathbf{X}_N \end{bmatrix} \\ &=\begin{bmatrix} \mathbf{X}_1^{\top} & \mathbf{X}_2^{\top} & \cdots & \mathbf{X}_N^{\top} \end{bmatrix} \begin{bmatrix} \mathbf{\Omega}^{-1} & \mathbf{0} & \cdots & \mathbf{0} \\ \mathbf{0} & \mathbf{\Omega}^{-1} & \cdots & \mathbf{0} \\ \vdots & \vdots & \ddots & \vdots \\ \mathbf{0} & \mathbf{0} & \cdots & \mathbf{\Omega}^{-1} \end{bmatrix} \begin{bmatrix} \mathbf{X}_1 \\ \mathbf{X}_2 \\ \vdots \\ \mathbf{X}_N \end{bmatrix} \\ &=\begin{bmatrix} \mathbf{X}_1 \\ \mathbf{X}_2 \\ \vdots \\ \mathbf{X}_N \end{bmatrix}^{\top} \begin{bmatrix} \mathbf{\Omega}^{-1} & \mathbf{0} & \cdots & \mathbf{0} \\ \mathbf{0} & \mathbf{\Omega}^{-1} & \cdots & \mathbf{0} \\ \vdots & \vdots & \ddots & \vdots \\ \mathbf{0} & \mathbf{0} & \cdots & \mathbf{\Omega}^{-1} \end{bmatrix} \begin{bmatrix} \mathbf{X}_1 \\ \mathbf{X}_2 \\ \vdots \\ \mathbf{X}_N \end{bmatrix} \\ &=\mathbf{X}_\bullet^{\top}\left(\mathbf{I}_N \otimes \mathbf{\Omega}^{-1}\right)\mathbf{X}_\bullet \end{align} where "$\otimes$" denotes the Kronecker Product.

Answer 2

Put $\mathbf{X}_i$'s next to each other to form a matrix. Oh, in fact you did. Yes, it is $X$.

Notice that $$\mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i$$ is the $i$'th element on the diagonal of

$$\mathbf{X}^{\top}\mathbf{\Omega}^{-1}\mathbf{X}$$

So, since you're summing these up, $$\begin{equation} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i \end{equation} = trace(\mathbf{X}^{\top}\mathbf{\Omega}^{-1}\mathbf{X}).$$ Done!