I am asking myself this question: Can anything be said of the resulting gradient, when the gradient of a function is approximated numerically (for example using the finite differences method) at a nondifferentiable point?
Could this result in any number? For example, when considering a convex function, will this always approximate some subgradient?
I wasn't able to find anything on this question, is the answer totally obvious?