$\newcommand{\tr}{\text{Tr}}$ Let us denote $L^2(X, \mathbb C^{n \times n})$ be the space of matrix-valued $L^2$ functions. That is, if $f \in L^2(X, \mathbb C^{n \times n})$, each entry of $f$ is an $L^2$ function, i.e., $f_{ij} \in L^2(X, \mathbb C)$ for $i,j \in \{1, \dots, n\}$. Suppose we have a fixed $g \in L^2(X, \mathbb C^{n \times n})$, we define a function $F: L^2(X, \mathbb C^{n \times n}) \to \mathbb R$ by $$ f \mapsto \text{Tr}(f^*g) \mapsto \left( \int|\text{Tr}(f^*g))|^2 dx \right).$$
Is $F$ Frechet differentiable? I did following computation \begin{align*} F(f+h) - F(f) &= \int \tr(g^*(f+h)) \tr((f+h)^*g) - \int \tr(g^*f)\tr(f^*g) \\ &=\int \tr(g^* h)\tr(f^*g) + \int\tr(g^*h)\tr(h^*g) + \int \tr(g^*f)\tr(h^*g). \end{align*} Now letting $h \to 0$ in $L^2$, I am not sure the quotient \begin{align*} \lim_{\|h\|_{L^2} \to 0} \frac{|F(f+h) - F(f)|}{\|h\|_{L^2}} \end{align*} converges to something.