Gradients of a loss function $ \| Y - X^{(K+1)} \|_F^2$ with recursive equation update described by $X^{(K+1)}$.

85 Views Asked by At

Problem definition:

The loss function (with Frobenius norm) is \begin{align} L := \| Y - X^{(K+1)} \|_F^2 \ , \end{align} where $Y, X^{(K)} \in \mathbb{R}^{m \times n}$. The superscript notation of $X^{(K+1)}$ shows that $K$ updates of a recursive equation are considered.

The recursive equation update is \begin{align} X^{(k+1)} := X^{(k)} + \Lambda^{(k)} \ \odot \left[ f_1\left( \alpha f_2 \left( X^{(k)}\right) + \alpha A^{(k)} - X^{(k)}\right) - \left( f_2\left(X^{(k)}\right) + A^{(k)} \right)\right], \\ k = 1,2,\ldots,K \end{align} where $\Lambda^{(k)} \in \mathbb{R}^{m \times n} $, and we can assume that both $f_i: \mathbb{R}^{m \times n} \mapsto \mathbb{R}^{m \times n}; \ i = 1,2$ are differentiable. The scalar $\alpha \in \mathbb{R}$ is given.

Question:

Computation of gradients, i.e., $\frac{\partial L}{\partial \Lambda^{(k)}}$ and $\frac{\partial L}{\partial A^{(k)}}$ in the above recursive equation. We can assume $K=2$.