I have a function $f:\mathbb{R}^n\to\mathbb{R}^m$. My goal is to bound the first order Taylor approximation of $f$. Given $x,x'\in\mathbb{R}^n$ I have that \begin{equation} f(x)-f(x')\approx (x-x')^TDf(x')+(x-x')^THf(x')(x-x')+\mathcal{O}\left(\left\Vert x-x' \right\Vert^3\right), \end{equation} where $Df(x')$ and $Hf(x')$ define the Jacobian and Hessian w.r.t $x'$, respectively. Taking the 2-norm of each side we get
\begin{align} \left\Vert f(x)-f(x')-(x-x')^TDf(x')\right\Vert_2&\leq \left\Vert(x-x')^THf(x')(x-x')\right\Vert_2\\ &\leq\left\Vert H f(x')\right \Vert_2\left\Vert x-x'\right \Vert^2_2 \quad (\text{operator norm}).\\ &\leq C \left\Vert x-x'\right \Vert^2_2\\ \end{align}
My goal is to bound (and find $C$) such that $\left\Vert Hf(x') \right \Vert_2\leq C$. The confusion I have is that as $f:\mathbb{R}^n\to\mathbb{R}^m$, the Jacobian and Hessian are both tensors. Is the correct approach to split $f=(f_1,f_2,\dots,f_n)$ and take $n$ separate Taylor approximations of $f_j:\mathbb{R}\to \mathbb{R}^m$ for $j=1:n$ and then we would have $\left\Vert Hf(x') \right \Vert_2^2=\sum_{j=1}^n\left\Vert Hf_j(x') \right\Vert_2^2$ where $Hf_j(x')$ is the individual Hessian of $f_j$ at the point $x'$ ?
Or am I missing something blindingly obvious here?
Thanks in advance.
I prefer to use this form of taylor ormula $f(x+h)=\sum_{k=1}^N \frac{1}{k!}\sum_{1\le i_1 ... i_k \le n}\frac{\partial^k f}{\partial x_{i_1}...\partial x_{i_k}}(x)h_{i_1}...h_{i_k} + o(\|h\|^N)$. Note than $\frac{\partial^k f}{\partial x_{i_1}...\partial x_{i_k}}(x)\in \mathbb R^m$. So for $\sum_{i,j=1}^n \frac{\partial^k f}{\partial x_i \partial x_j}(x) h_i h_j$, its norm $\le \max_{i,j}\|\frac{\partial^k f}{\partial x_i\partial x_j}(x)\|_{2,\mathbb R^m} (\max_{1...n}|h_i|)^2$, and $\max_{1...n}|h_i|\le C(n) \|h\|_{2,\mathbb R^n}$.