Let $f(x) = \max_i a_i^\top x$ be a polyhedral function with $x,a_i\in R^n,$ for all indices $i$. I want to understand $\nabla^2f(x).$
Let us assume that $x$ is such that there is a unique index $j=\arg\,\max_i a_i^\top x$, so that $f(x) = a_j^\top x$. It can be proved that $\nabla f(x) = a_j$. Then my question is the following:
Is it true that $\nabla^2f(x) = O$?
My rationale is the following: if I choose a vector $h$ with $\|h\|_2$ small enough, I can have $f(x+2h) = a_j^\top (x+2h)$, and $f(x+h) = a_j^\top (x+h)$, basically small perturbation of $x$ will not change the maximizer index $j$. Then the second-order partial derivatives should be $0.$ However, I am skeptical about this proof and it seems there might be a subtle issue that will prevent this to happen, but cannot place my finger on it right now. Can anybody please help on this issue? Thanks in advance.