I am trying to solve the following exercise for my Machine Learning course.
Expand this expression so that there are only quadratic terms:
$(\mathbf{x} - \mathbf{\mu})^T \mathbf{\Sigma}^{-1} ( \mathbf{x} - \mathbf{\mu} ) + (\mathbf{\mu} - \mathbf{\mu_0})^T \mathbf{S}^{-1} ( \mathbf{\mu} - \mathbf{\mu_0} ) $
Where $\mathbf{x},\mathbf{\mu},\mathbf{\mu}_0$ are vectors and $\Sigma^{-1}$ and $\mathbf{S}^{-1}$ are square, invertible matrices.
Now, doing some research, I found out that the quadratic form is a matrix that can be produced in the form $\mathbf{x}^T\mathbf{A}\mathbf{x}$
Unfortunately after a few operations, the expression cannot be made better than this (is that correct?):
$ \mathbf{x}^T\mathbf{\Sigma}^{-1}\mathbf{x} - \mathbf{x}^T\mathbf{\Sigma}^{-1}\mathbf{\mu} - \mathbf{\mu}^T\mathbf{\Sigma}^{-1}\mathbf{x} + \mathbf{\mu}^T\mathbf{\Sigma}^{-1}\mathbf{\mu} + \mathbf{\mu}^T\mathbf{S}^{-1}\mathbf{\mu} - \mathbf{\mu}_0^T\mathbf{S}^{-1}\mathbf{\mu} + \mathbf{\mu}_0^T\mathbf{S}^{-1}\mathbf{\mu}_0 - \mathbf{\mu}^T\mathbf{S}^{-1}\mathbf{\mu}_0 $
As you can see it's not just quadratic terms. Vectors, such as $\mathbf{x}$, $\mathbf{\mu}$ and $\mathbf{\mu}_0$, are sometimes both present in terms that are not in quadratic form. What I am missing?
(a naive side question: since the quadratic form is used to represent non-linear expression, can this still be called linear algebra?)