We have a system of linear equations $f_1 ...f_d$ in $\mathbb{R}^d$, such that $f_i(x_1 , x_2 ,..., x_d)=0$ for $i=1,...,d$ (click image for main question) Question
I have managed to do the first part of this question, but for bounding $||R||_2$ I ended up with a double sum of all the elements of $(\textbf{x} - \textbf{y})$ being multiplied together which is clearly a lot larger than $||\textbf{x} - \textbf{y}||_2^2$ like this: My attempt after using the Hessian and Taylor series.
Any help would be appreciated.
We have \begin{align*} f_{i}(y)=f_{i}(y)-f_{i}(x)=D^{1}f_{i}(x)[y-x]+\dfrac{1}{2}D^{2}f_{i}(z_{i})[y-x,y-x],~~~~x=(x_{1},...,x_{d}), \end{align*} and hence \begin{align*} f(y)&=(f_{1}(y),...,f_{d}(y))\\ &=D^{1}f(x)[y-x]+\dfrac{1}{2}\left(D^{2}f_{1}(z_{1})[y-x,y-x],...,D^{2}f_{d}(z_{d})[y-x,y-x]\right), \end{align*} where $D^{1}f(x)$ is the canonical matrix which rows are defined by $D^{1}f_{i}(x)$, $i=1,...,d$ respectively.
Now \begin{align*} &\left\|\left(D^{2}f_{1}(z_{1})[y-x,y-x],...,D^{2}f_{d}(z_{d})[y-x,y-x]\right)\right\|_{2}\\ &=\left(\sum_{i=1}^{d}\bigg|D^{2}f_{i}(z_{i})[y-x,y-x]\bigg|^{2}\right)^{1/2}\\ &\leq\left(\sum_{i=1}^{d}\left\|D^{2}f_{i}(z_{i})[y-x]\right\|_{2\rightarrow 2}^{2}\|y-x\|_{2}^{2}\right)^{1/2}\\ &\leq\left(\sum_{i=1}^{d}\|D^{2}f_{i}(z_{i})\|_{2\rightarrow 2}^{2}\|y-x\|_{2}^{4}\right)^{1/2}\\ &=\left(\sum_{i=1}^{d}\|D^{2}f_{i}(z_{i})\|_{2\rightarrow 2}^{2}\right)^{1/2}\|y-x\|_{2}^{2}, \end{align*} where \begin{align*} \|D^{2}f_{i}(z_{i})\|_{2\rightarrow 2}^{2}\color{red}\leq\sum_{m,n=1}^{d}\left|\dfrac{\partial f_{i}}{\partial x_{m}\partial x_{n}}(z_{i})\right|^{2}\leq K^{2}d^{2}. \end{align*}
Note that $D^{1}f_{i}(x)=\left(\dfrac{\partial f_{i}}{\partial x_{1}}(x),...,\dfrac{f_{i}}{\partial x_{d}}(x)\right)$, $D^{1}f(x)$ is the matrix which the first row is $D^{1}f_{1}(x)$,..., and the last row is $D^{1}f_{d}(x)$.
$D^{2}f_{i}(z_{i})$ is the Hessian matrix, $D^{2}f_{i}(z_{i})[y-x,y-x]$ is like $[y-x]^{T}D^{2}f_{i}(z_{i})[y-x]$.
Actually we have for a square matrix $A$, $\|Ax\|_{2}\leq\displaystyle\sum_{i=1}^{d}|x_{i}|\|Ae_{i}\|_{2}\leq\left(\sum_{i=1}^{d}\|Ae_{i}\|_{2}^{2}\right)^{1/2}\|x\|_{2}=\left(\sum_{i=1}^{d}\sum_{j=1}^{d}|a_{ij}|^{2}\right)^{1/2}\|x\|_{2}$, so $\|A\|_{2\rightarrow 2}^{2}\leq\displaystyle\sum_{i,j=1}^{d}|a_{i,j}|^{2}$.