circular differentiation

Question

Suppose one starts with a function $f: \mathbb R^2 \rightarrow \mathbb R$ using $\mu, \sigma^2$ as its input, i.e. $f=f(\mu, \sigma^2)$. (Note that here I omitted the specific form of $f$ since I think it is not really relevant.) Now consider the reparameterization by defining $\displaystyle \psi=e^{\mu+\frac{\sigma^2}{2}}$. That is, repalce all $\mu$ by $\mu=\ln\psi-\frac{\sigma^2}{2}$. Now the function has $\psi$ and $\sigma^2$ as input. That is, $f=f(\psi, \sigma^2)$. The question is to find partial derivatives of this function under the new parameterization with respect to $\psi$. It is a straightforward exercise to start with by treating $\sigma^2$ as a constant. And I will have $\frac{\partial f}{\partial \psi}$. However, I got confused when someone asked me whether it is correct to treat $\sigma^2=2(\ln\psi-\mu)$ as a function of $\psi$ and then calculating $\frac{d \sigma^2}{d \psi}$ wherever possible using product rule and chain rule. I think that is is not right in that it will result in some kind of circular calculation since now you might also treat $\mu$ as a function of $\psi$, as well. Can anyone see what is wrong with treating $\sigma^2$ as a function of $\psi$, please? Thank you!

Answer 1

Your confusion likely stems from using the symbol $f$ to represent two different functions. Let $y = f( \mu, \sigma^2)$ and $y = g( \psi, \sigma^2)$. Notice that it's the same quantity $y$, but the functions are different. Now, if we name one more function to convert between $\mu$ and $\psi$, namely $$ \psi = h(\mu, \sigma) = e^{\mu + \frac{\sigma^2}{2}}, $$ then we have the composition $f = g \circ h$. The chain rule now applies, in Leibniz notation, where products of derivatives look like "canceling fractions:" $$ \begin{align} \frac{dy}{d\mu} &= \frac{dy}{d\psi} \cdot \frac{d\psi}{d\mu} + \frac{dy}{d\sigma} \cdot \frac{d\sigma}{d\mu} \\ &= \frac{dy}{d\psi} \cdot \frac{d\psi}{d\mu}, \end{align} $$ assuming that $\sigma$ doesn't depend on $\mu$.