What is the derivative of the following?:
$ \frac{\partial( \hat{\mathbf{y}} - \mathbf{y})^{\circ2}}{\partial \hat{\mathbf{y}}}$
here $\mathbf{y}$ is a constant column vector of $n\times 1$ dimension. $\hat{\mathbf{y}}$ is another column vector, with the same dimension.
This is what I've been trying so far:
from rules of Hadamard derivatives:
$= 2(\hat{\mathbf{y}}-\mathbf{y}) \odot \frac{\partial( \hat{\mathbf{y}} - \mathbf{y})}{\partial \hat{\mathbf{y}}}$
$= 2(\hat{\mathbf{y}}-\mathbf{y}) \odot (\frac{\partial \hat{\mathbf{y}}}{\partial \hat{\mathbf{y}}} - \frac{\partial {\mathbf{y}}}{\partial \hat{\mathbf{y}}})$
$= 2(\hat{\mathbf{y}}-\mathbf{y}) \odot (\frac{\partial \hat{\mathbf{y}}}{\partial \hat{\mathbf{y}}} - 0)$
$= 2(\hat{\mathbf{y}}-\mathbf{y}) \odot (\mathbf{I})$ (a vector's derivative of itself is the identity matrix)
Now, I am not sure how to do the Hadamard product between the vector($n \times 1$), and the matrix $(n \times n)$ works - and was wondering whether I did something incorrectly.
Define the variable $$\eqalign{ x &= {\hat y}-y &\implies dx &= d{\hat y} \cr X &= {\rm Diag}(x) \cr w &= x\odot x \cr }$$ Calculate the differential and gradient of that last variable. $$\eqalign{ dw &= 2x\odot dx = 2X\,dx = 2X\,d{\hat y} \cr \frac{\partial w}{\partial{\hat y}} &= 2X = 2\,{\rm Diag}\big({\hat y}-y\big) \cr }$$ The trick is to replace the Hadamard product by the normal matrix product with a diagonal matrix.