Theorem 5.1: Let $A \subseteq \mathbb{R}^m$, $f: A \to \mathbb{R}^n$ differentiable at $a$. Then all the directional derivatives of $f$ at $a$ exist and $f'(a,u) = Df(a).u$
The proof Munkres provides starts in the following way:
Put $B:= Df(a)$. Fix $u \in \mathbb{R}^n, u \neq 0$. By hypothesis:
$$\lim_{t \to 0} \frac{f(a+tu) - f(a)-B.tu}{\Vert tu\Vert } = 0$$
How does this limit identity follow?
I know that $\lim_{h \to 0} \frac{f(a+h) - f(a) - Bh}{\Vert h \Vert} =0$.
Does it follow from the fact that we can interchange a limit and a continuous function?
More concretely, put $B(h) = \begin{cases} \frac{f(a+h) - f(a) - Bh}{\Vert h \Vert} \quad h \neq 0 \\ 0 \quad h = 0\end{cases}$
Then $B$ is continuous at $0$, and since $tu \to 0$ when $t \to 0$, we have
$$\lim_{t \to 0} B(tu) = B(\lim_{t\to 0}tu) = B(0) = 0$$
Is this the correct reasoning or am I missing something?
Yes, your reasoning is correct. More precisely, this follows from composition of limits: if $B(u) \to 0$ when $u \to 0$, then $B((u(t)) \to 0$ when $t \to 0$ for any function $t \mapsto u(t)$ such that $u(t) \to 0$ when $t \to 0$.