How to compute Frechet Derivatives that involve matrices.

Question

Suppose I have a linear operator which maps $S^n\to\mathbb R^{n \times n}$. It is represented as $LX:= XA^T + AX.$ How can I find the Frechet derivative of $f(X) = XA^T + AX.$

Answer 1

I will give the same proof included in https://math.stackexchange.com/a/3956281/864940.

The definition of Frechet derivative is the following

$\boldsymbol{Definition:}$ Let $\boldsymbol{V}$, $\boldsymbol{W}$ be vector spaces with norms $||.||_V, ||.||_W$. A function $f:\boldsymbol{V} \rightarrow \boldsymbol{W}$ is called Frechet differentiable at $\hat{x} \in \boldsymbol{int\ dom}f$, if $\exists$ a bounded linear operator $Df(\hat{x}): \boldsymbol{V} \rightarrow \boldsymbol{W}$ s.t., $$\lim_{x\rightarrow \hat{x}\\x \in \boldsymbol{int\ dom}f\\x \neq \hat{x}} \frac{||f(x) - f(\hat{x}) - Df(\hat{x})(x-\hat{x})||_W}{||x - \hat{x}||_V} = 0$$

If $f:\boldsymbol{V} \rightarrow \boldsymbol{W}$ is Frechet differentiable at $\hat{x}$, then $Df(\hat{x})$ is called the Frechet derivative of $f$ at $\hat{x}$.

$\boldsymbol{Claim}:$ Let $f:\boldsymbol{S}^n \rightarrow \boldsymbol{R}^{n \times n}$ be the given function such that $f(X) = AX +XA^T$. Then, Frechet derivative $Df(\hat{X})$ of $f$ is the linear operator given by $Df(\hat{X}) = f$, i.e., $$ Df(\hat{X})(X) = AX + XA^T$$

$\boldsymbol{Proof}:$ Let $ Df(\hat{X})(X) = AX + XA^T$, and consider the following limit, $$\lim_{X\rightarrow \hat{X}\\X \in \boldsymbol{int\ dom}f\\X \neq \hat{X}} \frac{||f(X) - f(\hat{X}) - Df(\hat{X})(X-\hat{X})||}{||X - \hat{X}||} = \lim_{X\rightarrow \hat{X}\\X \in \boldsymbol{int\ dom}f\\X \neq \hat{X}} \frac{||(AX + XA^T) - (A\hat{X} + \hat{X}A^T) - Df(\hat{X})(X-\hat{X})||}{||X - \hat{X}||} = $$ $$\lim_{X\rightarrow \hat{X}\\X \in \boldsymbol{int\ dom}f\\X \neq \hat{X}} \frac{||[A(X-\hat{X}) + (X-\hat{X})A^T] - [A(X-\hat{X}) + (X-\hat{X})A^T]||}{||X - \hat{X}||} = \lim_{X\rightarrow \hat{X}\\X \in \boldsymbol{int\ dom}f\\X \neq \hat{X}} \frac{||0||}{||X - \hat{X}||} = 0$$ Since the above limit goes to zero with the given operator, it is the Frechet derivative of the function $f$ at $\hat{X}$, i.e., $Df(\hat{X}):\boldsymbol{S}^n \rightarrow \boldsymbol{R}^{n \times n}$ such that $Df(\hat{X})(X) = AX + XA^T$. $\blacksquare$

Answer 2

The best linear approximation to a function that is already linear is the function itself. So $DL(X) = L$.