Why is the sum of outer products equal to the matrix product of a matrix and its transpose? So $A^TA = \sum_{i=1}^n a_i a_i^T$, where $A = [ a_0, a_1 , ... , a_n ] $, $a_i \in \mathbb{R}^k$. An answer or a link to the answer would be greatly appriciated. It seems so simple but I just cannot figure it out.
It is essentially this question of which I do not understand the answer: Matrix $A^T A$ as sum of outer products
I think you got the order wrong. $ A A^T = \sum_{i=1}^n a_i a_i^T $. What you should do is to is to look at a sequence of examples: $$ \left( \begin{array}{c c c} 1 \\ 2 \\ 3 \end{array} \right) \left( \begin{array}{c c c} 1 \\ 2 \\ 3 \end{array} \right) ^T = $$
$$ \left( \begin{array}{c c c} 1 & -1\\ 2 & -2\\ 3 & -3 \end{array} \right) \left( \begin{array}{c c c} 1 & -1\\ 2 & -2\\ 3 & -3 \end{array} \right) ^T = $$ Maybe you will see the pattern.
Then, look at $$ \left( \begin{array}{c c c} a_{1,1} \\ a_{2,1} \\ a_{3,1} \end{array} \right) \left( \begin{array}{c c c} a_{1,1} \\ a_{2,1} \\ a_{3,1} \end{array} \right) ^T = $$ $$ \left( \begin{array}{c c c} a_{1,1} & a_{1,2} \\ a_{2,1} & a_{2,2}\\ a_{3,1}& a_{3,2} \end{array} \right) \left( \begin{array}{c c c} a_{1,1} & a_{1,2} \\ a_{2,1} & a_{2,2}\\ a_{3,1}& a_{3,2} \end{array} \right) ^T = $$ and see if you get the pattern from that.