In a problem Spivak mentions that $f'(x^2)$ means the derivative of $f$ at the number which we happen to be calling $x^2$; it is not the derivative at $x$ of the function $g(x) = f(x^2)$.
This sentence really struck me because it means that I have been doing all of the Spivak's previous problems in a wrong way.
For example, when doing the problem: prove that $\lim_{x\to 0^+} f(x) = \lim_{x\to 0} f(x^2)$, I used the following: $f(x^2) = (g \circ h) (x)$, where $g(x) = f(x)$ and $h(x) = x^2$.
Why am I so shocked: I always used this notation and I was always able to prove everything I needed - I have never found a contradiction.
So, how to think about $f(x^2)$ ? And if it really just means that it is the value of the function evaluated at some number $x^2$ then why write it like a square at all (I assume that probably to say that the numbers considered are positive) and how to interpret $\lim_{x\to 0} f(x^2)$?
$f'(x^2)$ is indeed not the same as $(f \circ h)'(x)$. Its maybe easier to be confused if you use $\frac{d}{dx}$ notation, and write $f(x)$ with $x$ 'indeterminate' to mean the function $f = f(x)$, giving (in general) the non-identity $$ \frac{df}{dx}(x^2) \neq \frac{d}{dx} (f(x^2))$$
In the case of the limit $\lim_{x→ 0} f(x^2)$ there is no issue because there is no way to ambiguously bracket the expression.