How to compute the Gateaux's derivative of a smooth but not analytic function?

Question

Let $f(x)=\begin{cases}x^2~\mbox{if $x\geq 0$}\\ 0~\mbox{otherwise}\end{cases}$

Let $A, B$ be two Hermitian matrices. We treat $f$ as a matrix function where $f(A)$ can be obtained from $A$ by applying $f$ to all the eigenvalues of $A$.

My question is if the following derivative well defined. If yes, how to compute it?

$\frac{d}{dt}f(A+tB)|_{t=0}$

Answer 1

$f$ is not smooth but only $C^1$.

1. Two easy cases: $A>0$. If $U$ is in a neighborhood of $A$, then $f(U)=U^2$ and $Df_A(B)=AB+BA$.

$A<0$. If $U$ is in a neighborhood of $A$, then $f(U)=0$ and $Df_A=0$.

EDIT. Finally, I have done all calculation again and I was wrong (there is a stupid mistake in my programming).

2. In the other cases, $f$ has also a derivative. More generally, when $f\in C^1$, then $f$ has a derivative on the vector-space of symmetric matrices (and even on the normal matrices); beware: the result is false over $M_n$.

I give some examples: If $A\geq 0$, then $Df_A(B)=AB+BA$.

If $A\leq 0$, then $Df_A=0$.

The most difficult case is when $A$ has $>0$ eigenvalues AND $<0$ eigenvalues. Clearly, we may assume that $A$ is diagonal.

For example, let $A=diag(\lambda_1,\lambda_2)=diag(-1,2)$. Then the $n^2=4$ eigenvalues of $Df_A$ are the $\dfrac{f(\lambda_i)-f(\lambda_j)}{\lambda_i-\lambda_j},(i,j)$, if $\lambda_i\not= \lambda_j$ and, otherwise $f'(\lambda_i)$; we obtain the symmetric matrix $A'=\begin{pmatrix}0&4/3\\4/3&4\end{pmatrix}$.

Finally, $Df_A(B)=A'\circ B$, where $"\circ"$ is the Hadamar product.

For the details, you can see: Higham, Functions of matrices, Theorem 3.11, p. 62.