Minimization of quadratic function

57 Views Asked by At

For the quadratic function $\mathcal{E}(\textbf{w}) = \frac{1}{2}\sum_{i=1}^{N}(d_{i} - \textbf{w}^T\textbf{x}_{i})^{2} + \frac{\lambda}{2} \|\textbf{w} \|_{2}^{2}$, I was trying to obtain the $\hat{\textbf{w}}$ that minimizes it. So,

$$ \frac{\partial\mathcal{E}(\textbf{w})}{\partial\textbf{w}} = \sum_{i=1}^{N}(d_{i} - \textbf{w}^T\textbf{x}_{i})(-\textbf{x}_{i})+\lambda\textbf{w} = 0 $$ Hence, $$ \sum_{i=1}^{N}d_{i}\textbf{x}_{i} = \sum_{i=1}^{N} (\textbf{w}^T\textbf{x}_{i})\textbf{x}_{i} +\lambda\mathbb{I}\textbf{w} $$ At this point, I do not know how to put the right side on the form: $(R+\lambda\mathbb{I})\textbf{w}$, where $R$ is the cross correlation matrix and $\mathbb{I}$ is the identity matrix.

1

There are 1 best solutions below

3
On BEST ANSWER

Hint.

$$ \sum_{i=1}^{N} (\textbf{w}^T\textbf{x}_{i})\textbf{x}_{i}=\left(\sum_{i=1}^{N} \textbf{x}_i\textbf{x}_i^T\right)\textbf{w} = R\textbf{w} $$

then

$$ \textbf{w} =\left(R+\lambda\mathbb{I}\right)^{-1}\sum_{i=1}^{N}d_{i}\textbf{x}_{i} $$