In a recent exercise, our professor wrote that
$t^TXw = w^T X^T t$
where $t,w$ are vectors and $X$ is a matrix.
I tried it for a simple example and the identity seems to hold true. However, I can't find any reference to this online, so it made me curious why this works.
Does it work if $t$ and/or $w$ are matrices? Is the identity true only because there are 3 factors? Et cetera.
In general, for any two matrices that can be multiplied, we have this little theorem: $$ (AB)^T = B^T A^T $$ This has been applied twice to prove the thing your professor wrote. Note that $t^T X w$ is a real number, so its transpose is equal to itself.
\begin{align} t^T X w&= (t^T X w)^T\\ &= (t^T (Xw))^T \\ &= (Xw)^T (t^T)^T & \text{by the little theorem, with $A = t^T, B = Xw$} \\ &= w^T X^T (t^T)^T & \text{by the little theorem, with $A = w, B = X$} \\ &= w^T X^T t & \text{because transposing twice gives back the original matrix} \\ \end{align}
Proof of the little theorem:
The $ij$ entry of $AB$ is $\sum_k a_{ik} b_{kj}$; the $ij$ entry of $(AB)^T$ is therefore $\sum_k a_{jk} b_{ki}$
The $ik$ entry of $B^T$ is $b_{ki}$; the $kj$ entry of $A^T$ is $a_{jk}$. So the $ij$ entry of $B^T A^T$ is $$ \sum_k b_{ki} a_{jk}, $$ and this is evidently the same thing we computed above. QED.