I want to solve this problem using Taylor expansions.
I tried
\begin{align*} 1 - \cos (x) = 1 - \left( \sum_{k=0}^{n} (-1)^k \frac{x^{2k}}{(2k)!} +R_{2n}(x) \right) \;, \end{align*}
where $R_{2n}(x)$ is the remainder function.
For $n=2$, this yields
\begin{align*} 1 - \cos (x) = 1 - \left( 1 - \frac{x^2}{2} + \frac{x^4}{24} + R_4(x) \right) = \frac{x^2}{2} - \frac{x^4}{24} - o(x^4) \;. \end{align*}
Is it now possible to just state
\begin{align*} |o(x^4)| \leq \frac{x^4}{24} \; , \end{align*} and hence \begin{align*} \frac{x^2}{2}-\frac{x^4}{24}-o(x^4)\leq \frac{x^2}{2} \leq \frac{x^2}{2} + \frac{x^3}{6} \; ? \end{align*}
I think I found an appropriate method.
The formula for the remainder term of an $n$-degree Taylor polynomial around $a$ is given by
\begin{align*} R_n(x) = \frac{f^{(n+1)}(c)}{(n+1)!} (x-a)^{n+1} \; , \end{align*}
for some constant $c$ in the open interval between $a$ and $x$, so $c\in(a,x)$ or $c\in(x,a)$.
The remainder term for the second-order Taylor polynomial $T_2(x)$ of $\cos x$ is given by\begin{align*} R_2(x) = \frac{-\cos c}{6} x^3 \; , \end{align*}
which can be bounded: $-x^3/6 \leq R_2(x) \leq x^3/6$.
From this, the claim now immediately follows
\begin{align*} 1 - \cos x = 1 - (T_2(x) + R_2(x)) = 1 - (1 - \frac{x^2}{2} + R_2(x)) = \frac{x^2}{2} - R_2(x) \leq \frac{x^2}{2} + \frac{x^3}{6} \;. \end{align*}