ML vs Math terminology

58 Views Asked by At

I read often in machine learning the termonilogy "smooth" and "differentiable" referring to a property to commonly used activation functions. I really struggle sometimes, because for example the ReLU function is not differentiable, about the smoothness I'm really not sure either. I think in math the term "smooth" is commonly used to describe the class of differentiable functions, in other context (for example one of my references in differential geometry) they use the terminolgy "sufficiently smooth", meaning the mappings analyzed are $C^k$ differentiable, for some $k$ (commonly $2$). So to me "smooth" means we have existance of the derivatives at least to a certain order.

My question is... are both "differentiable" and "smooth" miss-used in the ML language (comparing to math terminology)?