The multivariate chain rule can be succinctly expressed as $$\frac{df}{dy} = \frac{\partial f}{\partial y} + \frac{\partial f}{\partial z}\frac{dz}{dy}$$ if $f = f(y, z)$ is a function of $y$ and $z$, and $z=z(y)$ a function of $y$.
Now, suppose many more functional dependencies are involved. For simplicity, suppose we're try to derive $\frac{dh}{dx}$, where $h=h(f, g)$, $g=g(f, x)$ and $f=f(x)$. The functional dependencies between these variables can be expressed by the following graph,
Gathering the change in each variable due to the one(s) preceding it, the desired derivative can be expressed as, $$ (\star) \quad \frac{dh}{dx} = \frac{\partial h}{\partial g}\cdot\frac{\partial g}{\partial x} + \frac{\partial h}{\partial g}\cdot\frac{\partial g}{\partial f}\cdot\frac{df}{dx} + \frac{\partial h}{\partial f}\cdot \frac{df}{dx} $$
One could interpret $(\star)$ as a summation over all paths from $x$ to $h$ in the dependency graph, where each path contributes the product of partial derivatives along its edges. e.g., the path $x\to f\to g\to h$ contributes the expression $\frac{\partial h}{\partial g}\cdot\frac{\partial g}{\partial f}\cdot\frac{df}{dx}$ to the summation at $(\star)$.
Questions
- Is the identity $(\star)$ indeed correct?
- Having in mind more complicated dependency graphs, can a derivative between two nodes indeed be considered as a summation over all paths connecting them, as above? It could be nice to have proof, if so.