Derivative of $F^TF$ with respect to $F$

Question

I stacked with evaluating the derivatives $$ \frac{\partial\left(F^TF\right)}{\partial F}~~ {\rm and}~~ \frac{\partial^2\left(F^TF\right)}{\partial F^2} $$ in terms of $F\in\mathbb{R}^{3\times3}$. Any hint or any idea would be acknowledged.

Answer 1

The best way to approach this sort of problem is to use differentials rather than the chain rule.

This is because the chain rule requires intermediate quantities which are 3rd & 4th order tensors, which are difficult to work with, whereas differentials are ordinary vector and matrix quantities.

For convenience, let $$\eqalign{M=F^TF-I\cr\cr}$$ Use the Frobenius (:) Inner Product and this new variable to rewrite the function and find its differential and gradient $$\eqalign{ W &= \frac{1}{2}\|M\|^2 \cr &= \frac{1}{2}M:M \cr\cr dW &= M:dM \cr &= M:\big(\,dF^TF+F^TdF\,\big) \cr &= (FM^T+FM):dF \cr &= 2\,FM:dF\cr &= 2\,(FF^TF-F):dF \cr\cr \frac{\partial W}{\partial F} &= 2\,(FF^TF-F) \cr\cr }$$ Now let's proceed from the gradient to the the hessian (second derivative). Note that the gradient is a matrix, $$G=2\,(FF^TF-F)$$ so the hessian will be a 4th order tensor $${\mathbb H} = \frac{\partial G}{\partial F}$$ Let's start with the differential of $G$ $$\eqalign{ dG &= 2\,(dF\,F^TF+F\,dF^TF+FF^TdF-dF) \cr\cr }$$ To simplify further, we need 2 special (isotropic) tensors $({\mathbb E},{\mathbb B})$ with components $$\eqalign{ {\mathbb E}_{ijkl} &= \delta_{ik}\,\delta_{jl}\cr {\mathbb B}_{ijkl} &= \delta_{il}\,\delta_{jk}\cr }$$ Then $$\eqalign{ dG &= 2\,({\mathbb E}F^TF+F{\mathbb E}F^T:{\mathbb B}+FF^T{\mathbb E}-{\mathbb E}):dF \cr {\mathbb H} &= 2\,({\mathbb E}F^TF+F{\mathbb E}F^T:{\mathbb B}+FF^T{\mathbb E}-{\mathbb E}) \cr\cr }$$ Finally, you wished to evaluate the hessian at $F=I$, which simplifies things greatly $$\eqalign{ {\mathbb H} &= 2\,({\mathbb E}+{\mathbb E}:{\mathbb B}+{\mathbb E}-{\mathbb E}) \cr &= 2\,({\mathbb E}+{\mathbb B}) \cr }$$

Answer 2

Merging of two answers:

First a remark about derivatives in the algebra $M_3$ of matrices of size $n\times n$.

You need to give a meaning to $\frac{\partial F}{\partial F}$. The upper $F$ is a function of the $n^2$ elements of $F$ and you are calculating a derivative w.r.t. all the $n^2$ elements of $F$. So a way to write is $ \frac{\partial F_{ij}}{\partial F_{kl}}=\delta_{ij,kl}$ which has $n^2\times n^2$ components. Now the derivative in the direction $H\in H$ is then $\left( \frac{\partial F}{\partial F} H \right)_{ij} = \sum_{kl} \delta_{ij,kl}H_{kl} = H_{ij}$ an extremely cumbersome way of writing. Should really be avoided if possible.

Setting $X=M_{3}({\Bbb C})$, $f:X \rightarrow X$, $f(F)=F^T F$: $$ f(F+H)= (F+H)^T (F+H) = F^T F + (F^T H + H^T F) + H^T H$$ which is identified with $f(F) + f'(F).H + 1/2 f''(F).(H,H) + \ldots$ where $f'(F).H = F^T H + H^T F$ is the 1st derivative (acting upon $H$) and $f''(F).(H,K) = H^T K + K^T H$ is the second derivative (symmetric bilinear form).

Now a shortcut (if you don't worry too much about rigour?) to get the derivative of a differentiable function $W$ at $F$ in a direction $H$ all you need to calculate: $\frac{\partial W(F)}{\partial F} (H)=\frac{d}{dt}_{|t=0} W(F+tH)$. Here $t$ is a just a usual real variable.

Let us take (from the commentary rather than the question, which should perhaps be reformulated?) $W(F)=\frac{1}{4} |F^T F-I|^2=\frac{1}{4} {\rm tr} ((F^T F-I)(F^T F-I))$ with $F\in X=M_{3}({\Bbb C})$. Let us find the derivative in a direction $H$: $$ \frac{\partial W(F)}{\partial F}(H)= \frac{d}{dt}_{|t=0} W(F+tH) = ... = \frac{1}{2} {\rm tr} ((H^T F + F^T H) (F^T F-I))$$

The second derivative of this expression in the direction $K$ is: $$ \frac{\partial^2 W(F)}{\partial F^2}(H,K)= \frac{d}{dt}_{|t=0}\frac{\partial W(F+tK)}{\partial F}(H) = ... = \frac{1}{2} {\rm tr} ((K^T H + H^T K) (F^T F-I))+ \frac{1}{2} {\rm tr}( (H^T F + F^T H)(K^T F + F^T K) )$$ Setting $F=I$ and $K=H$ the first term vanishes and the result is: $$ \frac{\partial^2 W(F)}{\partial F^2}(H,H)_{|F=I} =\frac{1}{2} {\rm tr}( (H^T + H)(H^T + H) ) = \frac{1}{2} |H^T+H|^2.$$ Thus, $$ \frac{\partial^2 W(F)}{\partial F^2}_{|F=I} (F,F) = \frac{1}{2} |F^T+F|^2$$

Answer 3

Hint: Expand using the product rule. It should be obvious how to simplify one of the resulting terms, so the problem will then be reduced to computing ${\partial(F^T)\over\partial F}$. One you have that, computing the second derivative should be easy.

As another hint, consider the analogous problem in elementary calculus. what would you guess the second derivative of $F^TF$ ought to be?