Given the second derivative of a function on $\Bbb R$ can be written as $$f''(x) = \lim_{h\to0} \frac{f(x+h) - 2f(x) + f(x-h)}{h^2}$$
then in analogy to how we generalize the first derivative, could we say a function $f: \Bbb R^m \to \Bbb R^n$ is twice differentiable at $x$ if there exists a bilinear function $B$ such that $$\lim_{h\to 0} \frac{\|f(x+h)-2f(x)+f(x-h)-B(h,h)\|_{\Bbb R^n}}{|\langle h,h\rangle|} = 0$$ where $\langle \cdot, \cdot\rangle$ is the Euclidean norm on $\Bbb R^m$? Then $B$ would be the second derivative?
The trouble with that approach is that $$ \lim_{h\to0} \frac{f(x+h) - 2f(x) + f(x-h)}{h^2} $$ may exist even if the function has no second derivative at $x$. Consider for example $f(x)=|x|\,x$ and $x=0$.