How to find the gradient for a forward-backward splitting problem

Question

I am struggling to work out how to find the gradient of the following function:

$g(\Phi_1,\Phi_2)$=$\frac 12$||z + $\tilde{D}_x$$\Phi_1$ + $\tilde{D}_y$$\Phi_2$||$^2$

It is needed for the forward-backward splitting problem

$\tilde{\Phi_1}$=$\Phi_1^i$-T$\nabla$$\Phi_1$$g(\Phi_1^i,\Phi_2^i)$

$\tilde{\Phi_2}$=$\Phi_2^i$-T$\nabla$$\Phi_2$$g(\Phi_2^i,\Phi_2^i)$

I have been told I need to use the vector chain rule

$h(x)=f(g(x))$, $h'(x)=g'(x)f'(g(x))$

$h(x)=f(Gx)$, $\nabla$$h(x)=G^T$$\nabla$f(Gx).

Any help at all would be hugely appreciated.

Answer 1

First of all, if $f(x) = h(Ax+b)$, then the chain rule tells us that $\nabla f(x) = A^T \nabla h(Ax+b)$. Furthermore, if $h(y) = \frac12 \| y \|_2^2$, then it is easy to show that $\nabla h(y) = y$. Combining these two facts, we see that the gradient of the function $f(x) = \frac12 \|Ax + b \|_2^2$ is $\nabla f(x) = A^T(Ax + b)$.

Now, for your problem, let $\Phi = \begin{bmatrix} \Phi_1 \\ \Phi_2 \end{bmatrix}$ and $D = \begin{bmatrix} \tilde D_x & \tilde D_y \end{bmatrix}$. Then $g(\Phi) = \frac12 \| z + D \Phi \|_2^2$, and so \begin{align} \nabla g(\Phi) &= D^T(z + D \Phi) \\ &= \begin{bmatrix} \tilde D_x^T \\ \tilde D_y^T \end{bmatrix}(z + \tilde D_x \Phi_1 + \tilde D_y \Phi_2 ) \\ &= \begin{bmatrix} \tilde D_x^T (z + \tilde D_x \Phi_1 + \tilde D_y \Phi_2 ) \\ \tilde D_y^T (z + \tilde D_x \Phi_1 + \tilde D_y \Phi_2 ) \end{bmatrix}. \end{align}