How to compute the derivative of $ \frac{\operatorname{diag}(\lambda) \cdot [e^{z_j}]}{\lambda^T \cdot [e^{z_j}]}$?

39 Views Asked by At

Let $\lambda \in \mathbb R^n$ be a constant. For convenience, we adopt the writing $z = [z_j]$ for $z \in \mathbb R^n$. After some computation, I obtain a function $f:\mathbb R^n \to \mathbb R^n$ such that

$$f(z) = \frac{\operatorname{diag}(\lambda) \cdot [e^{z_j}]}{\lambda^T \cdot [e^{z_j}]}.$$

My goal is to compute $\partial_z f$ in a neat way by using matrix calculus. I consulted the textbook Matrix Cookbook, but could not find relevant formula.

Could you please elaborate on how to compute $\partial_z f$?

1

There are 1 best solutions below

3
On BEST ANSWER

$\def\d{{\rm diag}}\def\D{{\rm Diag}}\def\p#1#2{\frac{\partial #1}{\partial #2}}$For ease of typing, define the variables $$\eqalign{ L &= \D(\lambda), \quad y &= \exp(z), \quad Y &= \D(y) \quad\implies\quad dy=Y\,dz \\ }$$ Write the function in terms of these new variables, then calculate its differential and gradient. $$\eqalign{ f &= \frac{Ly}{\lambda^Ty} \\ df &= \frac{L\,dy}{\lambda^Ty} - \frac{Ly\,(\lambda^Tdy)}{(\lambda^Ty)^2} \\ &= \frac{L\,dy}{\lambda^Ty} - \frac{f\lambda^Tdy}{\lambda^Ty} \\ &= \left(\frac{L-f\lambda^T}{\lambda^Ty}\right)dy \\ &= \left(\frac{L-f\lambda^T}{\lambda^Ty}\right)Y\,dz \\ \p{f}{z} &= \left(\frac{L-f\lambda^T}{\lambda^Ty}\right)Y \\ }$$