Setup:
Let $f:[a,b] \to \mathbb{R}$ be a function.
Let $g:[\alpha, \beta] \to \mathbb{R}$ be a continous function.
Suppose $x_0$ is an interior point of the domain of $g$ which gets mapped to an interior point of the domain of $f$. Suppose also that $g$ is differentiable at $x_0$ with $g'(x_0)=0$.
In the above setup, if we further assume that $f$ is differentiable at $g(x_0)$, then we know by the chain rule that $f \circ g$ is differentiable at $x_0$ with $(f \circ g)'(x_0)=f'(g(x_0))g'(x_0)=0$.
In general, however, $f$ might be not differentiable at $g(x_0)$. I want to use upper and lower derivatives to try to generalize the above idea. For now, let's focus on what can be said about the upper derivative of $f \circ g$ at $x_0$.
Are the assumptions in the above setup strong enough to guarantee that the upper derivative of $f \circ g$ evalutated at $x_0$ is $0$?
What if we assume additionally that the upper and lower derivatives of $f$ are bounded on $( \alpha, \beta)$?
Let $g:[-1,1]\to\mathbb R$ be given by $g(x)=x^2$. Let $f:[-1,1]\to\mathbb R$ be given by $f(x)=\sqrt{|x|}\,\sin\tfrac1{x}$, $f(0)=0$. Note that $f$ is continuous at $0$.
We have $$ \frac{f(g(h))-f(g(0))}h=\frac{|h|\sin\tfrac1{h^2}}h. $$ Then the upper derivative is $1$ and the lower derivative is $-1$. Indeed, we can take $h_n=\sqrt{\frac2{\pi+4\pi n}}$ and then $h_n\to0$ while $f(g(h_n))=1$ for all $n$; and if $h_n=\sqrt{\frac2{3\pi+4\pi n}}$ then $f(g(h_n))=-1$ for all $n$.
We can push this by taking instead $f(x)=|x|^{1/4}\,\sin\frac1x$, same $g$ as above, and now we get that the upper derivative is $+\infty$ while the lower derivative is $-\infty$.
If instead you take $$f(x)=\begin{cases} 1,&\ x=0\\ \ \\ 0,&\ x\ne0\end{cases}$$ then the upper and lower derivative exist and agree (and are uniformly bounded) at all points other than $0$. But at $0$ neither derivative exist.