I'm just starting out learning about optimization.
Not sure if this makes sense but...
Given a variable $x$ that is passed to two separate functions, $f(x)$ and $g(x)$, why do we add the derivatives $\frac{df}{dx}$ and $\frac{dg}{dx}$ to compute the total derivative of $x$?
Why is this derivative equivalent to $\frac{dh}{dx}$ where $h(x) = f(x) + g(x)$, even though in the first case we're not adding the two functions?
Edit: I think in terms of optimization it might be clearer.
Suppose I have a matrix $W$ representing a set of weights that I multiply by an input matrix $X$ to get $WX=Y$. The $w_{11}$ element in $W$ "affects" every value in the first row of $Y$ and these are all separate outputs; the values of multiple "separate functions" that I was talking about. We sum $\frac{\partial Y_{11}}{\partial w_{11}} + \frac{\partial Y_{12}}{\partial w_{11}} + \frac{\partial Y_{13}}{\partial w_{11}} + ...$ to get $\frac{\partial Y}{\partial w_{11}}$.
I was wondering why we sum these even though the values of the first row of $Y$ are not added together like in the example of $h(x)=f(x)+g(x)$.