Checking derivative of norm squared of a linear transformation

Question

Given $f: L(\mathbb{R}^n, \mathbb{R}^k) \to \mathbb{R}$ where $f(A)=|Ax|^2$ and fix $x\in \mathbb{R}^n$ then find $\partial_H f(A)$ where $H\in L(\mathbb{R}^n, \mathbb{R}^k)$.

My attempt: $$ \partial_H f(A) = \lim_{t\to 0}\left(\frac{|(A+t\cdot H)x|^2 - |Ax|^2)}{t}\right)\\ = \lim_{t\to 0}\left(\frac{\sum_{j=1}^k\left(\sum_{i=1}^n (a_{ji} + t\cdot h_{ji})x_i\right)^2-\sum_{j=1}^k\left(\sum_{i=1}^n a_{ji}x_i\right)^2}{t}\right)\\ = \lim_{t\to 0}\left(\frac{\sum_{j=1}^k\left(\sum_{i=1}^n a_{ji}x_i\right)^2 + 2t\sum_{j=1}^k\left(\sum_{i=1}^n (a_{ji}x_i) \sum_{i=1}^n(h_{ji}x_i)\right) + t^2\sum_{j=1}^k\left(\sum_{i=1}^n h_{ji} x_i\right)^2-\sum_{j=1}^k\left(\sum_{i=1}^n a_{ji}x_i\right)^2}{t}\right)\\ = \left(2\sum_{j=1}^k\left(\sum_{i=1}^n (a_{ji}x_i) \sum_{i=1}^n(h_{ji}x_i)\right)\right)\\ $$ however I can't seem to simplify this further.

Answer 1

Assume for the moment that the Frechet derivative $f'(A)$ exists. Then,

$\tag1f(A + v) = f(A) + f'(A)(v) + r(v)\ \text{where}\ \left|\frac{r(v)}{v}\right|\to 0\ \text{as}\ v\to 0.$

We have then

$\tag2\frac{f(A+tu)-f(A)}{t}=\frac{f'(A)(tu)+r(tu)}{t}\to f'(A)(u)\ \text{as}\ t\to 0.$

It follows that

$\tag3(Df_A)(u)=f'(A)(u)$

so it suffices to calculate $f'(A)(u).$

Now, $f$ is a compostion $A\overset g\mapsto Ax\overset h\mapsto |Ax|^2$ and so $f'(A)$ does exist and is given by

$\tag4 f'(A)(u)=h'(g(A))(g'(A))(u)=h'(g(A))(g(u))$

because $g$ is linear. Then,

$\tag5 h'(g(A))(g(u))=h'(Ax)(ux)=2\langle Ax,ux\rangle.$

Answer 2

Write the function in terms of the vector $y=Ax$ and calculate its differential. $$\eqalign{ f &= y^Ty \\ df &= 2y^Tdy = 2(Ax)^T(dA\,x) = 2x^TA^TdA\,x \\ }$$ Now substitute $\,dA=H\,dt\,$ to recover the desired derivative. $$\eqalign{ \partial_Hf &= \left(\frac{df}{dt}\right) = 2x^TA^THx \\ }$$