It is a basic matrix factorization problem. Suppose I have a matrix Y and it was factorized into low rank matrices W and H (rank is $d$, where $d$ <= $n$ ). The following is the noation:
$C \in \mathbb{R}^{m \times n}$ (it is a weight matrix), $Y \in \mathbb{R}^{m \times n}$, $W \in \mathbb{R}^{m \times d}$, $H \in \mathbb{R}^{n \times d}$, $W_{k.}$ is the $i$th row of $W$ (row vector of $W$),
The following is the objective function: $$f = \sum_{ij}C_{ij}(Y_{ij}- \sum_r W_{ir}H_{jr})^2$$
I would like to get the derivative of function $f$ with respect to each entry of $W$ (that is $W_{kl}$ (at this time $H$ is considered to be constant), that is:
$$\frac {\partial f}{\partial W_{kl}}=? \quad (1)$$
and the derivative of function f with respect to a row vector of $W$ (that is $W_{k.}$) (at this time $H$ is considered to be constant), that is: $$\frac {\partial f}{\partial W_{k.}}=? \quad (2)$$
What is the trick to get such derivative including double summation notation?
By the way, I can get the derivative $\frac {\partial f}{\partial W}$ (that is with respect to whole matrix $W$) using the hadamard noation and frobenius norm by convert the objective function to the form $f=\|C \circ(Y-WH^T)\|_F^2$ (I learned it from @hans and other people, thanks!). However using this method, some close form solution expression is not easy to get or the computation for the inverse of the final matrix is costly or the kronecker product will generate a bigger matrix which is difficult to store into the memory. That is the reason why I would like to get the derivative of function $f$ with respect to a row vector instead of the whole matrix and iterate computation to all rows.
For the double summation notation $\sum_{ij}$, what is the trick to remove or reduct double summation to single summation $\sum_j$? Or write down double summation from $\sum_{ij}$ to $\sum_i \sum_j$? I checked the similar question asked here, but it is difficult for me to master the technique of solving such derivative, so I sincerely hope you can help me!
I appreciate your help.