Starting from the Taylor theorem of order 2 for functions of several variables (a proof is tried here A proof of multivariable Taylor theorem of order 2) I would like to find what we call in french "un developpement limité d'ordre 2", I state this now :
Consider a function $f:A\subset\mathbb{R}^n\to\mathbb{R}$ that is $C^{2}$ on $N(X_0, r)\in A$. Then we have
$f(X_0 + Y) = f(X_0) + Df(X_0)Y + \frac{1}{2}Y^{t}D^{2}f(X_0)Y + \epsilon(X_0, Y)\lVert Y\rVert^{2}$ with $\lim_{Y\to 0_{n}}\epsilon(X_0, Y) = 0$. Thus we can put rewrite this as
$f(X_0 + Y) = f(X_0) + Df(X_0)Y + \frac{1}{2}Y^{t}D^{2}f(X_0)Y + o\left(\lVert Y\rVert^{2}\right)$ for $Y$ in a neighborhood of $0_n$.
The idea of the proof is the following : If $Y = 0_{n}$ we consider $\epsilon_{0}(X_0,Y) = 0$ which proves the result above. Otherwise (i.e. $\lVert Y\rVert\leq r$), consider $\epsilon(X_0,Y) = \frac{1}{\lVert Y\rVert^{2}}\left(f(X_0+Y) - f(X_0) - Df(X_0)Y - \frac{1}{2}Y^{t}D^{2}f(X_0)Y\right)$
The proof ends if we show that $\lim_{Y\to 0_{n}}\epsilon(X_0, Y) = 0$. Since $f$ is $C^{2}$ in the neighborhood of $X_0$ we can use Taylor theorem of order $2$ on $f(X_0 + Y)$ to get
$\epsilon(X_0,Y) = \frac{1}{\lVert Y\rVert^{2}}\left(f(X_0) + Df(X_0)Y + \frac{1}{2}Y^{t}D^{2}f(X_0+\theta Y)Y - f(X_0) - Df(X_0)Y - \frac{1}{2}Y^{t}D^{2}f(X_0)Y\right)$
$=\frac{1}{2\lVert Y\rVert^{2}}\left(Y^{t}D^{2}f(X_0+\theta Y)Y - Y^{t}D^{2}f(X_0)Y\right) = \frac{1}{2\lVert Y\rVert^{2}}Y^{t}\left(D^{2}f(X_0+\theta Y) - D^{2}f(X_0)\right)Y $
$ =\sum_{i=1}^{n}\sum_{j=1}^{n}\frac{y_i y_j}{2\lVert Y\rVert^{2}}\left(f_{x_i x_j}^{''}(X_0 + \theta Y) - f_{x_i x_j}^{''}(X_0)\right)$
It follows that (by the triangle inequality)
$\lvert\epsilon(X_0, Y)\rvert\leq\sum_{i=1}^{n}\sum_{j=1}^{n}\left\rvert f_{x_i x_j}^{''}(X_0 + \theta Y) - f_{x_i x_j}^{''}(X_0)\right\rvert$
which shows that $\lim_{Y\to 0_{n}}\epsilon(X_0, Y) = 0$ using the continuity of the second order partial derivatives at $X_0$
This seems correct to you ? And do you an elegant way to extend this at the order $p$ ? I thought of an induction, thank you a lot !