I'm doing some coursework on linear regression, and part of it requires finding a closed-form solution of the mean-squared error minimisation problem:
$$\min_{\bf w} \frac{1}{n} \sum_{i=1}^{n} (\mathbf{w}^T \mathbf{x}_i - y_i)^2$$
This is how I tried to do it (with some help from online sources):
$$ \frac{1}{n} \sum_{i=1}^{n} \nabla_{\bf{w}} (\mathbf{w}^T \mathbf{x}_i - y_i)^2 = 0 \\ \Rightarrow \frac{2}{n} \sum_{i=1}^{n}(\mathbf{w}^T \mathbf{x}_i - y_i) \mathbf{x}_i = 0 \\ \Rightarrow \sum_{i=1}^{n}(\mathbf{w}^T \mathbf{x}_i) \mathbf{x}_i = \sum_{i=1}^{n} y_i \mathbf{x}_i \\ \Rightarrow \sum_{i=1}^{n} \mathbf{x}_i \mathbf{x}_i^T \mathbf{w} = \sum_{i=1}^{n} y_i \mathbf{x}_i $$
My only issue here is rearranging the left-hand side to get from the second-last to the last line. Could someone show me how this is done? Thanks.
Notice that $\mathbf{a}_{i} := (\mathbf{w}^T \mathbf{x}_i)$ are scalars and so $\mathbf{a}_{i} = \mathbf{a}_{i}^T = (\mathbf{w}^T \mathbf{x}_i)^T = (\mathbf{x}_i^T \mathbf{w})$. By virtue of being scalars they also commute with the vectors, $\mathbf{x}_i$, giving us $$\mathbf{a}_{i} \mathbf{x}_i = \mathbf{a}_{i}^T \mathbf{x}_i = \mathbf{x}_i \mathbf{a}_{i}^T.$$ Hence $$ \sum_{i=1}^{n} y_i \mathbf{x}_i = \sum_{i=1}^{n}(\mathbf{w}^T \mathbf{x}_i) \mathbf{x}_i = \sum_{i=1}^{n} \mathbf{x}_i \mathbf{x}_i^T \mathbf{w}.$$