Multiplying Out Inner Products

Question

If I have a product of the form $(x-s)^tA(x-s)$ where $x$ and $s$ are vectors and $A$ is a matrix, how would I go about multiplying this out? Further, how would I go about taking its derivative?

Answer 1

For multiplying it out, you can use the fact that inner products are positive definite symmetric bilinear forms (the 'bilinear' bit being the key one here), and revert to inner product notation, using the definition that $\langle x,y\rangle =x^TAy$:

$\langle x-s,x-s\rangle = \langle x,x-s\rangle -\langle s,x-s\rangle = \langle x,x\rangle - \langle x,s\rangle - \langle s,x\rangle - \langle s,s\rangle $

And then use the fact that it's symmetric:

$=\langle x,x\rangle -2\langle x,s\rangle -\langle s,s\rangle $

and finally use the definition $\langle x,y\rangle =x^T Ay$ again:

$=x^TAx - 2(x^TAs) - s^TAs$

Regarding taking the derivative, note that $\frac{d}{dt}\langle x,y\rangle =\langle x',y\rangle +\langle x,y'\rangle $ (where the dash represents the derivative with respect to the single variable), and again use the symmetry of an inner product, and take it from there...

Hope that helps!

Answer 2

Considering a vector $x \in k^n$ as an $n \times 1$ matrix, you need to use various algebraic properties in the ring of matrices (distributivity, associativity, etc.) and the fact that $$ (x - s)^\top = x^\top - s^\top. $$