In what case Hessian matrix off-diagnal values are zero

Question

In the case of off-diagnal values of Hessian matrix are zero, what will be the implication?

I found a related question here.

If a function $f$ has the from $$f(x_1,x_2,x_3)=g_1(x_1)+g_2(x_2)+g_3(x_3)$$ then the second order derivative $$\frac{\partial^2}{\partial x_i \partial x_j} f(x) = 0 \qquad \forall i\neq j$$.

In the case of Hessian matrix, does it only implies the function is linear as @ErdelvonMises mentioned?

Answer 1

First recall the definition of the Hessian $H$ of a function $f$ $$ H_{i,j} = \partial_{x_i,x_j} f $$ For all the off-diagonal term to be zero (e.i. $H_{i,j} = 0 \;\;\;\; \forall ij : i \neq j$) $$ 0 = \partial_{x_i,x_j} f \;\;\;\; \forall ij : i \neq j$$ must happen, suppose $f$ is a linear combination of functions with dependency in no more than a variable $$ f = \sum_{m=1}^M a_m g_m(x_m) $$ so the $ 0 = \partial_{x_i,x_j} f \;\;\;\; \forall ij : i \neq j $ of $f$ is $$ \partial_{x_i,x_j} f = \sum_{m=1}^M a_m \partial_{x_i,x_j} g_m(x_m) = 0 $$ This trivially implies that if $f$ is a linear combination of functions with dependency in no more than a variable, then the off-diagonal entries will be zero. And I conjecture the before mentioned condition is nesesary and sufficient to the off-diagonal entriess of the Hessian be zero.

For gain some intuition of why it would no apply to other cases. Suppose $f$ is a multivariate polynomial of degree two $$ f = a + \sum_{n = 1}^N \sum_{m=1}^2 x_n^m + \sum_{n = 1}^N \sum_{m = 1}^M c_{n,m} x_n x_m $$ We again we get the Hessian off-diagonal derivatives $$ \partial_{x_i,x_j} f = \sum_{n = 1}^N \sum_{m = 1}^M c_{n,m} \partial_{x_i,x_j} x_n x_m \neq 0 \,\,\,\, \text{for} \,\, i \neq j$$

Answer 2

The other post doesn't actually solve the question as stated---let's do that. If you like you can appeal to the generalized Taylor expansion of $f$ to solve the problem immediately, or we can basically do the same thing with our "bare hands" below.

Suppose that $f$ is smooth. In showing that there are smooth functions $g_i$ on $\mathbb{R}$ such that $f(x_1, \ldots, x_n) = g_1(x_1) + \cdots + g_n(x_n)$, it will be slightly annoying that this decomposition is not unique if $n > 1$: e.g. we can always simultaneously replace $g_1$ with $g_1 - 1$ and $g_2$ with $g_2 + 1$ to get another distinct decomposition.

Instead let's impose the additional constraint that $g_i(0) = 0$ for all $i$ and that there exists a constant $c \in \mathbb{R}$ such that $f(x_1, \ldots, x_n) = c + g_1(x_1) + \cdots + g_n(x_n)$. We will show that such a decomposition exists and is unique when the Hessian of $f$ is diagonal.

Let's first guess the functions $g_i$: we better have that $$ g_i'(t) = \frac{d f}{d x_i}(0, \ldots, 0) + \int_0^t \frac{d^2 f}{d x_i^2}(0, \ldots, 0, t, 0, \ldots, 0) d t, $$ i.e. so that $$ g_i(t) := \frac{d f}{d x_i}(0, \ldots, 0) t + \int_0^t \int_0^s \frac{d^2 f}{d x_i^2}(0, \ldots, 0, s, 0, \ldots, 0) d s d t. $$ (Recall that we want $g_i(0) = 0$.) Then we also need $c := f(0, \ldots, 0)$.

Ok, so now put $h(x_1, \ldots, x_n) = f(0, \ldots, 0) + g_1(x_1) + \cdots + g_n(x_n)$: we claim $f = h$. Indeed, for any $(x_1, \ldots, x_n) \in \mathbb{R}^n$ by the ordinary 1-dimensional fundamental theorem of calculus we have $$ f(x_1, 0, \ldots, 0) - f(0, \ldots, 0) = \int_0^{x_1} \frac{d f}{d x_1}(t, 0, \ldots, 0) d t, $$ so that inductively we have \begin{align*} f(x_1, \ldots, x_n) - f(0, \ldots, 0) &= \int_0^{x_1} \frac{d f}{d x_1}(t, 0, \ldots, 0) d t + \int_0^{x_2} \frac{d f}{d x_2}(x_1, t, \ldots, 0) d t\\ &\phantom{===}+ \cdots + \int_0^{x_n} \frac{d f}{d x_n}(x_1, \ldots, x_{n - 1}, t) d t. \end{align*} If we can show that $\frac{d f}{d x_k}(x_1, \ldots, x_{k - 1}, t, x_{k + 1}, \dots, x_n) = \frac{d f}{d x_k}(0, \ldots, 0, t, 0, \dots, 0)$ for all $k$ then we will be done, since then the previous equation just becomes \begin{align*} f(x_1, \ldots, x_n) - f(0, \ldots, 0) &= \int_0^{x_1} g_1'(t) d t + \int_0^{x_2} g_2'(t) d t + \cdots + \int_0^{x_n} g_n'(t) d t\\ &= g_1(x_1) + \cdots + g_n(x_n), \end{align*} as desired.

To check that $\frac{d f}{d x_k}(x_1, \ldots, x_{k - 1}, t, x_{k + 1}, \dots, x_n) = \frac{d f}{d x_k}(0, \ldots, 0, t, 0, \dots, 0)$ we finally use the hypothesis that off-diagonal entries of the Hessian of $f$ vanish: note just like before by the fundamental theorem of calculus we have \begin{align*} \frac{d f}{d x_k}(x_1, \ldots, 0, t, 0, \dots, 0) - \frac{d f}{d x_k}(0, \ldots, 0, t, 0, \dots, 0) &= \int_0^{x_1} \frac{d}{d x_1} \frac{d f}{d x_k}(s, \ldots, 0, t, 0, \dots, 0) d s\\ &= 0, \end{align*} since $\frac{d}{d x_1} \frac{d f}{d x_k} = 0$ (because $k \not = 1$). so that inductively we can conclude \begin{align*} \frac{d f}{d x_k}(x_1, \ldots, x_{k - 1}, t, x_{k + 1}, \dots, x_n) - \frac{d f}{d x_k}(0, \ldots, 0, t, 0, \dots, 0), \end{align*} as desired.