I have recently come across various vector equations and their derivatives but I don't know how the derivatives have been calculated? Even on searching, I find the standard parametric vector equations, whose derivatives are computed independently. Can someone please help.
$$ f(x+\Delta (x)) = f(x) + c^T\Delta(x) + 0.5\Delta(x)^TH\Delta(x)$$
On using the optimal condition for classical Newton's Method, $i.e. \partial f/\partial \Delta(x) = 0$, we get $$\Delta x = -H^{-1}c$$ I don't understand the intermediate steps involved.
Someone else will probably provide a more erudite derivation, but my favourite way to find the derivative of a scalar product or matrix product is that of using vector components. For instance, the $i$ component of $\displaystyle{d\over dx}(a^Tx)$ is by definition $\displaystyle{\partial\over \partial x_i}(a^Tx)$, and $$ {\partial\over \partial x_i}(a^Tx)= {\partial\over \partial x_i}\sum_{k=1}^n a_kx_k= \sum_{k=1}^n a_k\delta_{ik}=a_i, $$ where $\delta_{ik}$ is the Kronecker symbol: $\delta_{ik}=1$ if $i=k$, $\delta_{ik}=0$ if $i\ne k$. That means that $\displaystyle{d\over dx}(a^Tx)=a$. In a similar way you can prove that $\displaystyle{d\over dx}(x^TMx)=2Mx$, if $M$ is a symmetric matrix.