Automatic Differentiation - Chain Rule Question

Question

I have the formula, and I was reading a basic implementation of automatic differentiation:

$$f\left(x\right) = x - \exp\left(-2\sin^{2}\left(4x\right)\right).$$

The evaluation trace for

$$f(\pi/16)$$ and $$f'(\pi/16)$$

where $f^{\prime}\left(x\right) = \dfrac{\partial f}{\partial x}$.

looks like this:

My question is, I do not understand how the derivatives are calculated (meaning the 3rd and fourth columns). Maybe I am missing something simply here, but the derivative of sin(4x) is not cos(x)(x), for example.

Answer 1

The third row, for example should be read as:

• Set $x_3 = \sin(x_2)$
• Then $\mathrm{d}x_3 = \cos(x_2) ~\mathrm{d}x_2$.
• Now combine it with the previous line where $x_2$ has been computed to equal $\frac{\pi}{4}$ and $\mathrm{d}x_2$ has computed to be equal to $4$, then $x_3 = \sin(\pi/4)$ and $\mathrm{d}x_3 = \cos(\pi/4) \cdot 4$.

(note that in automatic differentiation the object $\mathrm{d}x_2$ is not an infinitesimal differential, but an element of $T_{x_2} \mathbb{R}$, the tangent space of $\mathbb{R}$ at $x_2$.)

Another way of thinking about it is that the table is just a very verbose way of writing out the chain rule of differentiation; except unlike what you are familiar with doing in calculus class, it starts from the inside-most term and move steadily outward.