Directional derivative and Jacobian matrix

Question

I have a problem with an exercise that goes as follows:

Let $\mathbf{f}$ be a $\mathbb{R}^n\rightarrow \mathbb{R}^m$ function and $\mathbf{a}$ an interior point of the domain of $\mathbf{f}$. Let $\mathbf{v} \in \mathbb{R}^n$ and $\|\mathbf{v}\| = 1$. The directional derivative of $\mathbf{f}$ in $\mathbf{a}$ in the direction of $\mathbf{v}$ is given by $$ \frac{\partial \mathbf{f}}{\partial \mathbf{v}}(\mathbf{a}) := \lim_{t\to0, t\in\mathbb{R}} \frac{\mathbf{f}(\mathbf{a}+t \mathbf{v})-\mathbf{f}(\mathbf{a})}{t} \in \mathbb{R}^m. $$ Show that if $\mathbf{f}$ is differentiable in $\mathbf{a}$, then the directional derivate exists and is equal to $D\mathbf{f}(\mathbf{a})\cdot\mathbf{v}$, with $D\mathbf{f}(\mathbf{a})$ defined by: $$ D\mathbf{f}(\mathbf{a}) = \left( \begin{array}{ccc}{\frac{\partial f_{1}}{\partial x_{1}}(\mathbf{a})} & {\cdots} & {\frac{\partial f_{1}}{\partial x_{n}}(\mathbf{a})} \\ {\vdots} & {\ddots} & {\vdots} \\ {\frac{\partial f_{m}}{\partial x_{1}}(\mathbf{a})} & {\dots} & {\frac{\partial f_{m}}{\partial x_{n}}(\mathbf{a})}\end{array}\right). $$

Could anyone help me with this proof?

Answer 1

Since $f$ is differentiable at $a$, then,$$\lim_{w\to0}\frac{\bigl\lVert f(a+w)-f(a)-Df(a)(w)\bigr\rVert}{\lVert w\rVert}=0.\tag1$$So, consider the vectors $w$ of the form $tv$, with $t\in\mathbb R$. You deduce then from $(1)$ that$$\lim_{t\to0}\frac{\bigl\lVert f(a+tv)-f(a)-Df(a)(tv)\bigr\rVert}{\lVert tv\rVert}=0.\tag2$$Clearly, $(2)$ is equivalent to$$\lim_{t\to0}\frac{f(a+tv)-f(a)-tDf(a)(v)}t=0.$$In other words,$$\lim_{t\to0}\frac{f(a+tv)-f(a)}t=Df(a)(v).\tag3$$But the LHS of $(3)$ is the directional derivative of $f$ in the direction of $v$. And, since$$\begin{bmatrix}{\frac{\partial f_{1}}{\partial x_{1}}(a)} & {\cdots} & {\frac{\partial f_{1}}{\partial x_{n}}(a)} \\ \vdots & \ddots & \vdots \\ {\frac{\partial f_{m}}{\partial x_{1}}(a)} & \cdots & {\frac{\partial f_{m}}{\partial x_{n}}(a)}\end{bmatrix}$$is the matrix of $Df(a)$ with respect to the canonical basis, the RHS of $(3)$ is$$\begin{bmatrix}{\frac{\partial f_{1}}{\partial x_{1}}(a)} & {\cdots} & {\frac{\partial f_{1}}{\partial x_{n}}(a)} \\ \vdots & \ddots & \vdots \\ {\frac{\partial f_{m}}{\partial x_{1}}(a)} & \cdots & {\frac{\partial f_{m}}{\partial x_{n}}(a)}\end{bmatrix}.v.$$

Answer 2

This question, as is presented, is not possible to answer since you are being asked to prove something which is logically inconsistent - not very fair tbh.

The impossibility lies in the fact that there is no expression for $\mathbf{D}\mathbf{f}(\mathbf{a})\cdot\mathbf{v}$ - because there is no such thing as a dot product between a matrix and a vector. Furthermore, a dot product yields a scalar value, yet the directional derivative must be vector-valued in this instance (since $\mathbf{f}$ itself is given as being vector-valued, and the directional derivative is explicitly given to be a member of $\mathbb{R}^m$ by the definition from the question).

But we can easily show that the directional derivative is equal to $\mathbf{D}\mathbf{f}(\mathbf{a})\mathbf{v}$ - ie the matrix product of the Jacobian matrix $\mathbf{D}\mathbf{f}(\mathbf{a})$ (aka Derivative matrix) and the vector $\mathbf{v}$, as follows...

1. Firstly note that by setting $ \mathbf{c}(t)=\mathbf{a}+t \mathbf{v}$ the question-given definition for the directional derivative can be re-expressed: $$ \lim_{t\to0} \frac{\mathbf{f}(\mathbf{a}+t\mathbf{v})-\mathbf{f}(\mathbf{a})}{t} \equiv \lim_{t\to0} \frac{\mathbf{f}(\mathbf{c}(t))-\mathbf{f}(\mathbf{c}(0))}{t} \tag{1} $$
2. Now, we define $\mathbf{g}(t)=f(\mathbf{c}(t))$. Observe that we know $\mathbf{g}'(t)$ exists, by the Chain Rule, since it is the derivative of the composition of $\mathbf{f}$ (given as differentiable by the question) and $\mathbf{c}$ (also differentiable), and we proceed to evaluate this derivative at $t=0$:

$$ \mathbf{g}'(0)= \lim_{h\to0} \frac{\mathbf{g}(0+h)-\mathbf{g}(0)}{h}\\=\lim_{h\to0}\frac{\mathbf{f}(\mathbf{c}(h))-\mathbf{f}(\mathbf{c}(0))}{h} \equiv \lim_{t\to0} \frac{\mathbf{f}(\mathbf{c}(t))-\mathbf{f}(\mathbf{c}(0))}{t} \tag{2} $$ ...where the final line follows from the fact that changing the limiting variable from $h$ to $t$ is irrelevant in determining value of the limit.

3. By comparison of $(1)$ with $(2)$ we can see: $$ \begin{align} \lim_{t\to0} \frac{\mathbf{f}(\mathbf{a}+t\mathbf{v})-\mathbf{f}(\mathbf{a})}{t} =\mathbf{g}'(0) &=\frac{d}{dt}\Bigg|_{t=0}\mathbf{f}(\mathbf{c}(t)) \\ &=\mathbf{D}\mathbf{f}(\mathbf{c}(0))\mathbf{D}\mathbf{c}(0) &&\textit{ (Directly by the chain rule)}\\ &=\mathbf{D}\mathbf{f}(\mathbf{a})\mathbf{c}'(0) \\ &=\mathbf{D}\mathbf{f}(\mathbf{a})\mathbf{v} &&\textit{ ($\mathbf{c}'(0)=\mathbf{v}$)}\\ \\&\therefore \underline{\underline{=\frac{\partial\mathbf{f}}{\partial\mathbf{v}}(a) \equiv\mathbf{D}\mathbf{f}(\mathbf{a})\mathbf{v}}} \end{align} $$