This question follows from Calculating gradient for inner product of variable matrix which was very helpful. I am now trying to calculate the Hessian with respect to variable matrix U and have a quick follow up question. These are some of the basic forms defined in the previous question:
Using notation $$\langle A, B \rangle = A:B$$ Let $U\in\mathbb{R}^{d\times R}$ denote the variable matrix. $A_i\in\mathbb{R}^{d\times d}$ and $y_i$ is a scalar
$$v_i = A_i:(UU^T) - y_i$$
$$M = \frac 1m\sum_{i=1}^m v_iA_i$$
To solve for the Hessian(U), I need to find $dM$:
$$\eqalign{ dM &= \frac 1m\sum_{i=1}^m dv_iA_i \\ &= \frac 1m\sum_{i=1}^m dv_iA_i \\ &= \frac 1m\sum_{i=1}^m (A_i:d(UU^T))A_i \\ &= \frac 1m\sum_{i=1}^m (A_i:(dUU^T + UdU^T))A_i \\ }$$
Then I use the method of the 4th step in the gradient derivation in the linked question to get the next line:
$$\eqalign{ &= \frac 1m\sum_{i=1}^m A_i\,((A_i+A_i^T)U:dU) \\ }$$
My question is can I use the same method that was used in the previous question of sort of dividing the right side by $dU$ to get $\frac {dM} {dU}$? I am not sure because of the extra $A_i$ term.