Little notation question about matrix multiplications / quadratic forms

Question

I'm a bit confused about expanding out the notation of product of matrices, in the context of quadratic forms.

If $x \in \mathbb{R}^n, \, \, A \in \mathbb{R}^{n \times n}$ then

$$x^TAx = \sum_{i,j=1}^na_{ij}x_ix_j$$

But then if I consider a matrix $X \in \mathbb{R}^{n \times n}$ how should I write the expanded form of

$$X^TAX = \, \,...\, \, ?$$

This time the result will be a matrix.. will it be something like

$$X^TAX = \sum_{i,j=1}^n a_{ij}x_ix_j^T$$

and if yes, why?

Sorry if this is pretty straightforward but it always happens to get a little bit stuck with matrix notation.

Many thanks,

James

Answer 1

The $(i,j)$-coefficient of the matrix $X^TAX$ is given by \begin{equation} \sum_{k,l=1}^nx_{ki}a_{kl}x_{lj}, \end{equation} where $x_{ij}$ (resp. $a_{ij}$) is the coefficient of $X$ (resp. $A$).

Answer 2

It's easy to see why your attempt is wrong by considering a matrix $X$ of dimension $n\times p$. Then $X^TAX$ has dimension $p\times p$, while your formula yields a $n\times n$ matrix.

You can work out the contribution of $a_{i,j}$ to the product by writing the product:

$$X^TAX=\sum_{i,j,k,l} x_{i,k}a_{i,j}x_{j,l}e_{k,l}$$

Where $e_{k,l}$ are elementary matrices, and the sum runs on all valid indices. All elements of the matrix $e_{k,l}$ are $0$ except at index $(k,l)$ where it's $1$.

Then:

$$X^TAX=\sum_{i,j} a_{i,j} \left(\sum_{k,l} x_{i,k}x_{j,l}e_{k,l}\right)$$

And inside the parentheses we recognize $x_i^Tx_j$, where $x_i$ is the $i$th row.

If it's difficult to recognize, consider this: the element at index $(k,l)$ is $x_{i,k}x_{j,l}$. That is, the matrix $\sum_{k,l} x_{i,k}x_{j,l}e_{k,l}$ is an outer product $uv^T$ where $u$ and $v$ are column vectors. The $k$th element of $u$ must be $x_{i,k}$, that is $u$ has the same elements as the row $x_i$, except it's a column vector, so $u=x_i^T$. And the $l$th element of $v$ is $x_{j,l}$, and in the product you get $v^T$, which is a row vector, hence $v^T=x_j$. Therefore $\sum_{k,l} x_{i,k}x_{j,l}e_{k,l}=x_i^Tx_j$.