I'm new to Frechet and Gateaux differentiation, this likely has some mistakes. I was hoping someone could check this and offer some guidance.
Suppose we have the following functional: \begin{equation} F(\theta): \theta\in\Theta \rightarrow \exp[\frac{1}{a}\int_{t\in T}\int_{s\in T}(x(t)-\theta(t))K(t,s)(x(s)-\theta(s))] \end{equation}
Where $\Theta\subseteq L_2$, with $L_2$ the Hilbert space of Lebesgue measure $\mu$ square integrable functions on $T$, endowed with inner product $<f,g> = \int_{t\in T}f(t)g(t)dt$ and corresponding norm $||\circ|| = \sqrt{<\circ,\circ>}$. $x(t)$ is an arbitrary element of $L_2$, and $T \subseteq \mathbb{R}^n$ We assume $K$ is the integral kernel representation of a continuous, linear, symmetric, and positive definite operator, A, such that: \begin{equation} \int_{t\in T}\int_{s\in T}\phi(t)K(t,s)\phi(s) = <\phi,A\phi> \end{equation}
It can be shown that $A$ is differentiable. We have that $A$ is continuous and linear, thus:
\begin{equation} \begin{split} \lim_{||h||\rightarrow 0}\frac{||A(\phi + h) - A\phi - D[A](h)||_{L_2}}{||h||}& = \lim_{||h||\rightarrow 0}\frac{||A\phi + Ah - A\phi - D[A]h||_{L_2}}{||h||}\\ & =\lim_{||h||\rightarrow 0}\frac{||Ah - D[A](h)||_{L_2}}{||h||}\\ \end{split} \end{equation}
Thus for $D[A](h) = Ah$ we get that the limit is $0$ for all sequences of functions $||h_n|| \rightarrow 0$.
Similarly, the inner product $<\phi,A\phi>$ is Frechet differentiable for $\phi \in \Theta$. We have that:
\begin{equation} \begin{split} & \lim_{||h||\rightarrow 0}\frac{||<\phi + h, A(\phi+h)>-<\phi,A\phi> - D[<\phi,A\phi>](h)||}{||h||}\\ & = \lim_{||h||\rightarrow 0}\frac{||<\phi,A\phi> + <h,A\phi> + <\phi,Ah> + <h,Ah> - <\phi,A\phi> - D[<\phi,A\phi>](h)||}{||h||}\\ & = \lim_{||h||\rightarrow 0}\frac{||<h,A\phi> + <\phi,Ah> + <h,Ah> - D[<\phi,A\phi>](h)||}{||h||} \end{split} \end{equation}
So the derivative exists and is:
\begin{equation} D[<\phi,A\phi>](h) = <h,A\phi> + <\phi,Ah> + <h,Ah> \end{equation}
We know the derivative of the exponential function is well defined. So by the chain rule of Frechet differentiability, Theorem 6 here we have that the Frechet derivative exists. From Lemma 2.1 here we have that the Gateaux derivative exists and is equal to the Frechet derivative. We proceed to find the Gateaux derivative.
I noted that our situation fits the example outlined here, for: \begin{equation} \begin{split} H(\theta(t)) = \frac{1}{a}\int_{t \in T}\int_{s\in T}(x(t)-\theta(t))K(t,s)(x(s)-\theta(s)) \end{split} \end{equation}
Using the fact that $D[e^x](h) = he^{x}$, and that the chain rule gives us that $D[f\circ g] = D[f](D[g](h))$ I feel like the solution should be:
\begin{equation} D[\frac{1}{a}\int_{t \in T}\int_{s\in T} (x(t)-\theta(t))K(t,s)(x(s)-\theta(s))](h)\exp[\frac{1}{a}\int_{t \in T}\int_{s\in T} (x(t)-\theta(t))K(t,s)(x(s)-\theta(s))] \end{equation}
From the previously cited Wikipedia example:
\begin{equation} \begin{split} & D[\frac{1}{a}\int_{t \in T}\int_{s\in T} (x(t)-\theta(t))K(t,s)(x(s)-\theta(s))](h) \\ &= \frac{1}{a}\int_{t \in T}\int_{s\in T} D_\theta[(x(t)-\theta(t))K(t,s)(x(s)-\theta(s))]hdtds(h) \end{split} \end{equation}
This is where I'm stuck. I wasn't sure how to take the derivative of the integrand of the left factor with respect to $\theta$. The primary thing that is confusing me is the $\theta(t)\theta(s)$ term that comes out when you foil $(x(t)-\theta(t))(x(s)-\theta(s))$. It contains $\theta$, but for different arguments being integrated over, thus I'm not sure how to proceed. I did some manipulations to try and figure out if I could use Leibniz integral rule to get the integration, but wasn't able to find anything, so I decided not to put the calculations in here.
EDIT:
I also should note that I have this Theorem that I was considering using if I could satisfy the assumptions, but similarly was having issues understanding how to use it, and reconcile why it wasn't working out to be equal to the other method of using the Gateaux derivative given in the Wikipedia page.