How to determine if a (cost) function is convex?

Question

I am playing around with some machine learning models and want to try out different cost functions to see how the models react. One of the requirements for a cost function that can guarantee us to find a global minimum (using gradient descent) is that the cost function must be convex.

How do I determine whether a function I'm using is convex (in the realm of machine learning) and only has a global minimum?

Sorry if this doesn't make sense, I don't have a full understanding of the topic yet...

Answer 1

• Check whether it is convex from the definition.

• Check that that Hessian matrix is positive semidefinite.

• Also, multiplying a convex function by a positive constant, give you a convex function. Adding two convex functions give you a convex functions too. This is useful to see that some regularized objective functions are convex.