Hessian of distance between two points against coordinates

Question

I have two points in an N-dimensional space like this: $$\vec r_1 = (x_1,y_1,z_1,w_1,....) \text{ and } \vec r_2 = (x_2, y_2, z_2, w_2, ...)$$

I want to calculate the gradient and Hessian of the Euclidean distance between these two points, against all of the coordinates.

So, distance (d) is $$d = \sqrt{(x_1-x_2)^2 + (y_1 -y_2)^2+ \;...}$$

Now, the gradient is quite simple to calculate: $$\vec{\nabla} d = (\frac{\partial d}{\partial x_1},\frac{\partial d}{\partial y_1},\frac{\partial d}{\partial z_1},...,\frac{\partial d}{\partial x_2},\frac{\partial d}{\partial y_2},....)$$

The first part of this would be $1/d(\vec r_1 - \vec r_2)$ and the second would be $1/d(\vec r_2 - \vec r_1)$. Then I would concatenate these two arrays. This is quite simple to do with python (numpy).

However, I am unable to find a simple matrix/vector based expression for the Hessian matrix. In the Hessian matrix, there would be cross derivative terms ($\frac{\partial^2 d}{\partial x_1 \partial x_2}$).

How do I calculate the Hessian matrix using matrix or vector operations (that would be easy to do on code)?

(I need gradient and Hessian against all the coordinates, not just coordinates of one point)

Answer 1

I suggest writing the gradient in the order $\partial d/\partial x_1, \partial d/ \partial x_2,... $. Let $D=d^2$. So your gradient is $$[\frac{(x_1-x_2)}{D^{1/2}},\frac{-(x_1-x_2)}{D^{1/2}},\frac{(y_1-y_2)}{D^{1/2}},\frac{-(y_1-y_2}{D^{1/2}},...]$$. Now work out your Hessian. In your first row of your Hessian, you have the partial derivatives of $\frac{(x_1-x_2)}{D^{1/2}}$ with respect to the variables $x_1,x_2,y_1,y_2,...$. The first two items are easy applications of the quotient rule and and the next two are even easier since the numerator of the function you are differentiating does not involve $y_1$ or $y_2$. Your result will look prettiest if you express the Hessian in 2x2 blocks. Note that you can take out a factor of $1/D^{3/2}$

Answer 2

It is easy to see that $$\frac{\partial \phi}{\partial \mathbf{x}} =\frac{1}{d} (\mathbf{x-y}), \frac{\partial \phi}{\partial \mathbf{y}} =\frac{1}{d} (\mathbf{y-x})$$

Let $\mathbf{z}=\begin{pmatrix} \mathbf{x} \\ \mathbf{y} \end{pmatrix}$ so it follows that $$ \mathbf{g} =\frac{\partial \phi}{\partial \mathbf{z}} =\frac{1}{d} \begin{pmatrix} +\mathbf{I}&-\mathbf{I} \\ -\mathbf{I}& +\mathbf{I} \end{pmatrix} \mathbf{z} =\frac{1}{d} \mathbf{A} \mathbf{z} $$ From here $$d\mathbf{g}=\frac{1}{d} \mathbf{A} d\mathbf{z}- \frac{1}{d^2} \mathbf{Az} d\phi $$ The Hessian writes $$ \mathbf{H} = \frac{1}{d} \left[ \mathbf{A}- \frac{1}{d} \mathbf{Az} \mathbf{g}^T \right] = \frac{1}{d^3} \left[ d^2 \mathbf{A}- \mathbf{b} \mathbf{b}^T \right] $$ with $\mathbf{b}=\mathbf{Az}$.