Let $f:R\to R$, $g:R^n\to R$. Thus $f\circ g:R^n\to R$. Now suppose $f$ is non-decreasing and convex while $g$ is convex. In additon, $f,g$ are of $C^2$. I want to show that their composition is convex by proving its Hessian matrix is positive semi-definite.
I know there are easier way to prove convexity which does not even need $C^2$ by following definition. But I just want to know how to prove it using Hessian matrix.
I tried to write down each entry of the Hessian matrix using chain rule, but it turned out to be messy, and I don't see where to use their convexity and $f's$ non-decreasing property.
Any help would be appreciated.
You have $\nabla(f\circ g)(x) = f'(g(x)) \, \nabla g(x)$. Consequently $$\nabla^2(f \circ g)(x) = f''(g(x)) \, \nabla g(x) \, \nabla g(x)^\top + f'(g(x)) \, \nabla^2 g(x).$$ Then, it is easy to check the definiteness of the Hessian.