Derivative of dot product formula proof.

Question

Let $f: \mathbb{R}^n \to \mathbb{R}^m$ and $g: \mathbb{R}^n \to \mathbb{R}^m$

Define $h : \mathbb{R}^n \to \mathbb{R}$ as $h = f.g$

Suppose $f, g$ are differentiable at $y$ which is defined as:

$\forall \epsilon > 0$, $\exists \delta > 0$ such that $|x - y| < \delta$ $\implies $ $$\frac{\Big | f(x) - f(y) - df(y)(x - y) \Big |}{|x - y|} < \epsilon $$ for some $m \times n$ matrix $df(y)$.

Now, I want to prove $h$ is differentiable at $y$, i.e. there exists matrix $M(y) \in \mathbb{R}^{1 \times n}$ such that $$\frac{\Big |h(x) - h(y) - M(y)(x - y) \Big |}{|x - y|} < \epsilon \, \, \,\, \,\, \,\, \,\, \,\, \,\, \, (1)$$

Fix some $\epsilon> 0$

Let's consider $M(y) = g(y)^T df(y) + f(y)^T dg(y)$. Further define:

1. $F(x, y) = f(x) - f(y) - df(y)(x - y)$
2. $G(x, y) = g(x) - g(y) - dg(y)(x - y)$

Then, after a lot of simplification, I got that $|h(x) - h(y) - M(y)(x - y)|$ is equal to: $$\big |g(y)^T F(x, y) + f(y)^T G(x, y) + \left( g(x) - g(y) \right).\left( f(x)-f(y) \right) \big|$$

Then we have: \begin{align*} \frac{\big |g(y)^T F(x, y) + f(y)^T G(x, y) + \left( g(x) - g(y) \right).\left( f(x)-f(y) \right) \big|}{|x - y|} \end{align*} \begin{align*} &\leq |g(y)| \frac{|F(x, y)|}{|x - y|} + |f(y)| \frac{|G(x, y)|}{|x - y|} + \frac{|g(x) - g(y)| |f(x) - f(y)|}{|x - y|} \end{align*}

1. Since $f$ is differentiable at $y$, there exists $\delta_1 > 0$ such that $|x - y| < \delta_1$ $\implies $ $\frac{|F(x, y)|}{|x - y|} < \frac{\epsilon}{3 \, |g(y)|}$.
2. Since $g$ is differentiable at $y$, there exists $\delta_2 > 0$ such that $|x - y| < \delta_2$ $\implies $ $\frac{|G(x, y)|}{|x - y|} < \frac{\epsilon}{3 \,|f(y)|}$.
3. As $x \to y$, we have $\frac{|f(x) - f(y)|}{|x - y|} \to |f'(y)|$
4. Since $g$ is continuous at $y$, there exists $\delta_3 > 0$ such that $|x - y| < \delta_3$ $\implies $ $|g(x) - g(y)| < \frac{\epsilon}{3 |f'(y)|}$

Note that $y$ is fixed so the above 4 points make sense. Finally, choosing $\delta = \min \left(\delta_1, \delta_2, \delta_3 \right) $ and considering $|x - y| < \delta$ and further letting $x \to y$, we have that \begin{align*} |g(y)| \frac{|F(x, y)|}{|x - y|} + |f(y)| \frac{|G(x, y)|}{|x - y|} + \frac{|g(x) - g(y)| |f(x) - f(y)|}{|x - y|} < \epsilon \end{align*}

Answer 1

Ideally you should avoid $\epsilon$'s and $\delta$'s at all. Here's something useful:

Claim: If $B:\Bbb R^n \times \Bbb R^m\to \Bbb R^p$ is bilinear, then $B$ is differentiable and we have $$DB(x,y)(h,k)= B(x,k)+B(h,y).$$ Proof: Let $0<C =\sup\{\|B(u,v)\| : \|u\|=\|v\|=1\}<+\infty$, so that $B(h,k)\leq C\|h\|\|k\|$ for all $h\in \Bbb R^n$ and $k\in \Bbb R^m$. Then $$0\leq \left\|\frac{B(x+h,y+k)-B(x,y)-B(x,k)-B(h,y)}{\sqrt{\|h\|^2+\|k\|^2}}\right\|\leq \frac{C\|h\|\|k\|}{\sqrt{\|h\|^2+\|k\|^2}}\to 0$$as $(h,k)\to (0,0)$.

Now the dot product $B=\langle\cdot,\cdot\rangle$ is bilinear, and the chain rule kicks in for $h = B\circ (f,g)$ to give

$$\begin{align}Dh(x)(v)&= DB(f(x),g(x))(Df(x)(v),Dg(x)(v))\\ &= B(f(x),Dg(x)(v))+ B(Df(x)(v),g(x))\\ &= \langle f(x),Dg(x)(v)\rangle + \langle Df(x)(v),g(x)\rangle.\end{align}$$