Let a $\theta$ be a 2 Dimensional vector and $\alpha$, $\beta$ be positive scalars. Let $f(\theta)$ be a function given as:
$ f(\theta) = \alpha \exp(-\beta \: \theta^T \theta)$
I can find the gradient of the function using the chain rule but I am having some difficulty in finding the Hessian. The gradient which I computed is the following:
$\nabla f(\theta) = - 2 \alpha \beta \exp(-\beta \: \theta^T \theta) \:\theta$
What is the Hessian of $f(\theta)$. In other words what is the derivative of $\nabla f(\theta)$.
If $\theta = (x,y)$, then $$f(x,y) = \alpha e^{-\beta(x^2+y^2)}. $$ So $f_x(x,y) = -2\alpha \beta x e^{-\beta(x^2+y^2)}$, and so $$f_{xx}(x,y) =-2\alpha \beta e^{-\beta(x^2+y^2)} +4\alpha \beta^2 x^2 e^{-\beta(x^2+y^2)} = 2\alpha\beta (2\beta x^2-1)e^{-\beta (x^2+y^2)}.$$By symmetry: $$f_{yy}(x,y) = 2\alpha\beta (2\beta y^2-1)e^{-\beta (x^2+y^2)}.$$ Finally: $$f_{xy}(x,y)=4\alpha \beta^2 xye^{-\beta (x^2+y^2)}.$$So we conclude that $${\rm Hess}\;f (x,y) =2\alpha\beta e^{-\beta (x^2+y^2)}\begin{pmatrix} 2\beta x^2-1 & 2\beta xy \\ 2\beta xy & 2\beta y^2-1 \end{pmatrix}. $$Going back to $\theta$ notation, this writes as $${\rm Hess}\;f (\theta) = 2\alpha\beta e^{-\beta \theta^\top\theta} (2\beta \theta \theta^\top -{\rm Id}_2). $$