Find the Gradient of $\left \| Ax-b \right \|_2^2$ using Chain Rule

50 Views Asked by At

We have the Chain Rule for composite matrix-functions, that given dimensionally compatible functions, then
$\triangledown_Xg(f(X)^T)=\triangledown_Xf^T\triangledown_fg$
So how can I use this Chain Rule to compute the gradient of $f(x) = \left \| Ax-b \right \|_2^2$ without expanding the function like $f(x)=(Ax-b)^T(Ax-b)$?

1

There are 1 best solutions below

1
On

Let $h(x)=Ax-b$ and $g(z) = \|z\|^2$ so that $f = g \circ h$. Show that $\nabla g(z) = 2z$ and $\nabla h(x) = A$.