I compute Hessians on random samples of my non-convex high-dimensional optimisation problem to get a better understanding of the function shape. I thought I will quantify the amount of ill-conditioning by calculating the condition numbers of the Hessians at the sampled points. However, most of the time the eigenvalues are positive as well as negative, and some of the eigenvalues are zero. Does it mean that the condition number can not be computed? I have found the following statement on-line:
...There may be points during training where the Hessian is indefinite (i.e., has both positive and negative eigenvalues) and the condition number does not exist
Is it true? Or can I perhaps just calculate the absolute values of the eigenvalues to compute the ratio? Since condition number is max eigenvalue divided by min eigenvalue, I am assuming the answer will be infinity for zero eigenvalues. But what about the saddles?