A linear operator $\mathcal{L}_A : \mathcal{S}^n\to \mathbf{R}^{n\times n}$ is represented as $\mathcal{L}_AX:=AX+XA^{\top}$. Find a representation for Frechet derivative of $f(X) = AX+XA^{\top}$ and $g(X) = A^2X+2AXA^{\top}+X(A^{\top})^2$.
I am not sure if I understand the question clearly. What do I need to do exactly? Calculate the Frechet derivative of the given $2$ functions and then somehow represent them in terms of the linear operator? What does it exactly mean to represent in terms of an operator? I am new to this field of mathematics, so I am not sure how to go about the problem.
The definition of Frechet derivative is the following
$\boldsymbol{Definition:}$ Let $\boldsymbol{V}$, $\boldsymbol{W}$ be vector spaces with norms $||.||_V, ||.||_W$. A function $f:\boldsymbol{V} \rightarrow \boldsymbol{W}$ is called Frechet differentiable at $\hat{x} \in \boldsymbol{int\ dom}f$, if $\exists$ a bounded linear operator $Df(\hat{x}): \boldsymbol{V} \rightarrow \boldsymbol{W}$ s.t., $$\lim_{x\rightarrow \hat{x}\\x \in \boldsymbol{int\ dom}f\\x \neq \hat{x}} \frac{||f(x) - f(\hat{x}) - Df(\hat{x})(x-\hat{x})||_W}{||x - \hat{x}||_V} = 0$$
If $f:\boldsymbol{V} \rightarrow \boldsymbol{W}$ is Frechet differentiable at $\hat{x}$, then $Df(\hat{x})$ is called the Frechet derivative of $f$ at $\hat{x}$.
In the light of this definition, we can find the Frechet derivative of the given functions.
$\boldsymbol{Claim}:$ Let $f:\boldsymbol{S}^n \rightarrow \boldsymbol{R}^{n \times n}$ be the given function such that $f(X) = AX +XA^T$. Then, Frechet derivative $Df(\hat{X})$ of $f$ is the linear operator given by $Df(\hat{X}) = f$, i.e., $$ Df(\hat{X})(X) = AX + XA^T$$
$\boldsymbol{Proof}:$ Let $ Df(\hat{X})(X) = AX + XA^T$, and consider the following limit, $$\lim_{X\rightarrow \hat{X}\\X \in \boldsymbol{int\ dom}f\\X \neq \hat{X}} \frac{||f(X) - f(\hat{X}) - Df(\hat{X})(X-\hat{X})||}{||X - \hat{X}||} = \lim_{X\rightarrow \hat{X}\\X \in \boldsymbol{int\ dom}f\\X \neq \hat{X}} \frac{||(AX + XA^T) - (A\hat{X} + \hat{X}A^T) - Df(\hat{X})(X-\hat{X})||}{||X - \hat{X}||} = $$ $$\lim_{X\rightarrow \hat{X}\\X \in \boldsymbol{int\ dom}f\\X \neq \hat{X}} \frac{||[A(X-\hat{X}) + (X-\hat{X})A^T] - [A(X-\hat{X}) + (X-\hat{X})A^T]||}{||X - \hat{X}||} = \lim_{X\rightarrow \hat{X}\\X \in \boldsymbol{int\ dom}f\\X \neq \hat{X}} \frac{||0||}{||X - \hat{X}||} = 0$$ Since the above limit goes to zero with the given operator, it is the Frechet derivative of the function $f$ at $\hat{X}$, i.e., $Df(\hat{X}):\boldsymbol{S}^n \rightarrow \boldsymbol{R}^{n \times n}$ such that $Df(\hat{X})(X) = AX + XA^T$. $\blacksquare$
For the second function $g(X) = A^2X + 2AXA^T+X(A^T)^2$, the Frechet derivative at $\hat{X}$ is $Dg(\hat{X}) = g$, i.e., $$Dg(\hat{X})(X) = g(X) = A^2X + 2AXA^T+X(A^T)^2$$ The above proof can be applied to prove that.
$\boldsymbol{Remark}:$ The reason why $Df(\hat{X}) = f$, and $Dg(\hat{X}) = g$ is that the both functions are linear in $X$.