derivative of jacobian matrix concatenated with a vector function

Question

Let $f:\mathbb{R}^d \to \mathbb{R}^d, (x_1,...,x_d)\mapsto \left( \matrix{f_1(x_1,...,x_d) \\ \vdots\\ f_d(x_1,...,x_d)} \right)$ and $y:\mathbb{R} \to \mathbb{R}^d, t\mapsto \left( \matrix{y_1(t) \\ \vdots \\ y_d(t)} \right)$. Then $$ \frac{d(f \circ y)}{dt}(s) = \underbrace{\left( \matrix{(\frac{df_1}{dx_1}\circ y)(s),...,(\frac{df_1}{dx_d}\circ y)(s) \\ \vdots \\ (\frac{df_d}{dx_1}\circ y)(s), ... , (\frac{df_d}{dx_d} \circ y)(s)} \right)}_{=(\frac{df}{dx}\circ y)(s)} \underbrace{\left( \matrix{\frac{dy_1}{dt}(s)\\ \vdots \\\frac{dy_d}{dt}(s) } \right)}_{= \frac{dy}{dt}}~,$$ for $s \in \mathbb{R}$. What is $\frac{d}{dt}(\frac{df}{dx}\circ y)$? If you imagine $\frac{d^2f}{dx^2} \circ y$ as d jacobian matrices, how can you imagine the product with the derivative of the inner function $y$?

The solution is:

$\frac{d}{dt}(\frac{df}{dx} \circ y) = \frac{dy}{dt}^T \cdot \left( \left[\matrix{\frac{df}{dxdx_1} \circ y \\ \vdots \\ \frac{df}{dxdx_d} \circ y } \right] \cdot \left[ \frac{dy}{dt} \right] \right)$

Answer 1

$(f\circ y)'(t) = f'(y(t))[y'(t)]$, where $[v]$ means application of linear map to the vector $v$, and then $$(f\circ y)''(t)= f''(y(t))[y'(t),y'(t)] + f'(y(t))[y''(t)] $$ where $[v,w]$ means application of a bilinear map $f''(y(t))$ to vectors $v,w$.

If this is not useful for you, then nne way to avoid having to imagine anything is to write everything in coordinates. Rewriting the exact same thing: for each $i$, we have $$(f\circ y)_i'(t) = \sum_j \partial_j f_i(y(t))y'_j(t)$$ and then $$ (f\circ y)_i''(t)=\sum_{j,k} \partial_k\partial_jf_i(y(t))y'_j(t)y'_k(t)+\sum_j \partial_j f_i(y(t))y''_j(t)$$ having never left the very intuitive 1D world.

Answer 2

It is a case of the Chain's rule:

You have $\mathbb R\cdot\!\!\stackrel y\dashrightarrow\mathbb R^d\cdot\!\!\stackrel f\dashrightarrow\mathbb R^d$ in composition, , which in matrix terms is $$s\mapsto\left(\begin{array}{c} (f\circ y)^1\\(f\circ y)^2\\\vdots\\(f\circ y)^d \end{array}\right).$$ Now, your $f\circ y:\mathbb R\cdot\!\!\dashrightarrow\mathbb R^d$ should have derivative $(f\circ y)'=(f' \circ y) \cdot y'$. With the matrix version, that, would be
$$\left(\begin{array}{c} \dfrac{d(f\circ y)^1}{ds}\\\dfrac{d(f\circ y)^2}{ds}\\\vdots\\\dfrac{d(f\circ y)^d}{ds} \end{array}\right)= \left(\begin{array}{cccc} \dfrac{\partial f^1}{\partial y^1}&\dfrac{\partial f^1}{\partial y^2}&...&\dfrac{\partial f^1}{\partial y^d}\\ \dfrac{\partial f^2}{\partial y^1}&\dfrac{\partial f^2}{\partial y^2}&...&\dfrac{\partial f^2}{\partial y^d}\\ \vdots\\ \dfrac{\partial f^d}{\partial y^1}&\dfrac{\partial f^d}{\partial y^2}&...&\dfrac{\partial f^d}{\partial y^d}\\ \end{array} \right) \left(\begin{array}{c} \dfrac{d y^1}{ds}\\\dfrac{dy^2}{ds}\\\vdots\\\dfrac{dy^d}{ds} \end{array}\right). $$ From this, abbreviating $$(f\circ y)''=\underbrace{ \underbrace{(f'' \circ y)}_{\in \mathbb{R}^{d \times d \times d}} \cdot \underbrace{y'}_{\in \mathbb{R}^{d}} }_{\text{what is happening here?}} \cdot y' + (f' \circ y) \cdot y''.$$