Say I have the function $f(x) = |x|$
I believe that $x = 0$ is a critical point, although not I'm not positive. As the function is decreasing and increasing each side of $x = 0$ does that alone make it a local minimum even though at $x = 0$ the function is not differentiable?
The definition of "local minimum" does not depend on the function being differentiable.
$0$ is a local minimum because $f(0)=0$ and there is a neighborhood of $0$ (namely the entire real line) such that $f(x)>f(0)$ for all $x\ne 0$ in that neighborhood.
IF the function is differentiable at the point in question, then you can use the derivative to gather information about whether it has a chance of being a local extremum or not. However, if it is not differentiable, then that tells you nothing at all about local maxima and minima. There may be one, or there may not; you need to go back to the actual definition for that.