To me the following seems intuitively true:
If $f$ is differentiable at $x$ with surjective derivative then $g$ is differentiable at $f(x)$ iff $g\circ f$ is differentiable at $x$.
On the other hand, if $g$ is differentiable at $f(x)$ with injective derivative then $f$ is differentiable at $x$ iff $g\circ f$ is differentiable at $x$.
I assume $f,g$ to be mappings between Banach spaces but I guess this should not make a huge difference.
Useful fact 1: If $f$ is differentiable at $x$ with surjective derivative, then there is an open neighborhood $U$ containing $f(x)$ and a map $s$ on $U$ such that $f\circ s=id_U$.
Useful fact 2: If $g$ is differentiable at $y$ with injective derivative, then there is an open neighborhood $V$ containing $g(y)$ and a map $p$ on $V$ such that $p\circ g=id$.
These facts can be shown by the inverse function theorem.
We now show that the first condition holds. One direction is clear. Assume $g\circ f$ is differentiable at $x$, and let $s$ as in useful fact 1. Then locally we have$$g=g\circ id=g\circ (f\circ s)=(g\circ f)\circ s$$and hence $g$ is differentiable as a composition of differentiable maps.
The second condition can be proved similarly, using fact 2.