Let $\mathbf{W}(\alpha)$ be a matrix which depends to parameter $\alpha$ and let $\mathbf{f}$ be a vector. I want to approximate $\mathbf{W}(\alpha+\Delta \alpha)\mathbf{f}$ using Taylor expansion.
My work:
- First method:
$$ \mathbf{W}(\alpha+\Delta \alpha)\mathbf{f} \approx \bigg[\mathbf{W}(\alpha)+\mathbf{J}_{\mathbf{W}(\alpha)}(\alpha)\Delta \alpha\bigg]\mathbf{f} $$
$$= \mathbf{W}(\alpha)\mathbf{f}+\mathbf{J}_{\mathbf{W}(\alpha)}(\alpha)\Delta \alpha\mathbf{f} $$
- Second method: $$ \mathbf{W}(\alpha+\Delta \alpha)\mathbf{f} \approx \mathbf{W}(\alpha)\mathbf{f}+\mathbf{J}_{\mathbf{W}(\alpha)\mathbf{f}}(\alpha, \mathbf{f})\Delta \alpha $$
where $\mathbf{J}_{\mathbf{W}(\alpha)}(\alpha)$ is Jacobian matrix of $\mathbf{W}(\alpha)$ with respect to $\alpha$ and $\mathbf{J}_{\mathbf{W}(\alpha)\mathbf{f}}(\alpha, \mathbf{f})$ is Jacobian matrix of $\mathbf{W}(\alpha)\mathbf{f}$ with respect to $\alpha$.
My questions:
Are both methods correct?
Is there any preferences to choosing one of them?
Thanks.
Sure, both methods are correct. What you're seeing is that $$J_{W(\alpha)f}(\alpha, f) = J_{W(\alpha)}(\alpha)f.$$
It's hard to say if one method is better than the other, since they are so very similar; one difference is that in the first method, you have to compute the Jacobian of a matrix-valued quantity, and in the second, of a vector-valued one; for taking scalar derivatives it doesn't much matter, but if you were differentiating with respect to, say, a vector-valued parameter, the first method would require writing down an expression involving rank three tensors, while the second would just have matrices and vectors. This comes up when differentiation rotations: calculating $$\frac{d}{d\theta}R_\theta v$$ where $R_\theta v$ is the rotation of a vector $v$ by the axis-angle $\theta$, has a much nicer expression than the derivative $\frac{d}{d\theta}R_\theta$ of the rotation matrix itself.
By the way, if $A$ and $B$ are of any shape or size and $\odot$ is a bilinear operator, $$d(A\odot B) = dA \odot B + A \odot dB$$ where $d$ is the differential; in the special case where $A$ is a matrix, $B$ is a vector, $\odot$ is matrix-vector multiplication, $dB=0$, and the differential is with respect to a single scalar variable $\alpha$, you recover your formula.