Let $F, G: \mathbb{R}^{n \text{ x } n} \to \mathbb{R}^{n \text{ x } n}$ be two differentiable functions. Defining $(F \cdot\ G)(A):=F(A)G(A)$ using the usual matrix multiplication, prove that $F\cdot\ G$ is differentiable and express $(F\cdot\ G)'$ in terms of $F$, $F'$, $G$, $G'$.
What I thought of doing was to write $A=[a_{ij}]_{i, j \in {1, ..., n}}$ and express $F$ as a matrix $[f_{ij}]_{i, j \in {1, ..., n}}$ of differentiable functions $f_{ij}:\mathbb{R} \to \mathbb{R}$ (same idea for $G$), and I imagine $(F\cdot\ G)'$ will look something like $F'\cdot\ G+F\cdot\ G'$, but I don't know how to formalize it.
[obs.: the matrix norm considered here is $||A||=\sup_{|v|=1}|A(v)|$, where $v \in \mathbb{R^n}$ and $|\cdot\ |$ is the usual euclidean norm]
The guess was more or less in the right direction, but the meaning of $F'G+FG'$ needs to be stated precisely.
First, we should establish some notation: for $M_0 \in \mathbb{R}^{n\times n}$, the expression $F'(M_0)[M]$ means the derivative of $F$ at $M_0$, applied to $M$, for $M\in \mathbb{R}^{n\times n}$. Letting $M_0$ be fixed, we'll also use the notation $\Delta F(M):=F(M_0+M)-F(M_0)$.
Now what we will prove is this:
$$(F\cdot\ G)'(M_0)[M]=F'(M_0)[M]G(M_0)+F(M_0)G'(M_0)[M] \text{, for all }M\in\mathbb{R}^{n\times n}$$
(notice that $M$ - and every other matrix for that matter - needs to be in the right places, since matrix multiplication is not commutative)
By definition, we need to show that:
$$\lim_{||M||\to 0}\frac{||\Delta (F\cdot\ G)(M)-(F'(M_0)[M]G(M_0)+F(M_0)G'(M_0)[M])||}{||M||}=0 \text{ (*)}$$
Let's make some rearrangements. Notice that:
\begin{align*} \Delta (F\cdot\ G)(M) &=F(M_0+M)G(M_0+M)-F(M_0)G(M_0) \\ &=(\Delta F(M)+F(M_0))(\Delta G(M)+G(M_0))-F(M_0)G(M_0)\\ &=\Delta F(M) \Delta G[M] + \Delta F(M) G(M_0)+F(M_0)\Delta G(M) \end{align*}
Pluging this in (*) and rearranging the terms, what we actually need to show is:
$$\frac{||\Delta F(M)\Delta G(M)+(\Delta F(M)-F'(M_0)[M])G(M_0)+F(M_0)(\Delta G(M)-G'(M_0)[M])||}{||M||}\to 0$$
as $||M||\to 0$. Using the triangle inequality and the fact that $||A\cdot\ B||\leq ||A||\cdot\ ||B||$ we have that the last expression is less or equal to the sum:
$$\frac{||\Delta F(M)\Delta G(M)||}{||M||}+ \frac{||\Delta F(M)-F'(M_0)[M]||}{||M||}||G(M_0)||+||F(M_0)||\frac{||\Delta G(M)-G'(M_0)[M]||}{||M||}$$
Notice that since $F, G$ are differentiable, the last two terms go to zero, by definition of derivative. So we're only left to prove that the first term $\frac{||\Delta F(M)\Delta G(M)||}{||M||}$ goes to zero.
Since $\Delta F(M) = F'(M_0)[M] + o(||M||)$ and $\Delta G(M) = G'(M_0)[M] + o(||M||)$, we get:
$$\Delta F(M_0)\Delta G(M_0)= F'(M_0)[M]G'(M_0)[M]+o(||M||)$$
as $||M||\to 0$. Now, since $M$, $F'(M_0)$ and $G'(M_0)$ are linear operators themselves, we have:
\begin{align*} \lim_{||M||\to 0}\frac{||\Delta F(M_0)\Delta G(M_0)||}{||M||} &= \lim_{||M||\to 0}\frac{||F'(M_0)[M]G'(M_0)[M]||}{||M||} \\ &\leq \lim_{||M||\to 0}{\frac{||F'(M_0)[M]||}{||M||}||G'(M_0)[M]||}\\ &\leq \lim_{||M||\to 0}\frac{||F'(M_0)||\cdot\ ||M||}{||M||} ||G'(M_0)[M]||\\ &= ||F'(M_0)||\lim_{||M||\to 0}||G'(M_0)[M]|| \end{align*}
But linear operators are continuous, so $||G'(M_0)[M]||\to 0$ as $||M||\to 0$ and we're done.