The following curious approximation
$\cos\left ( \frac{\alpha}{3} \right ) \approx \frac{1}{2}\sqrt{\frac{2\cos\alpha}{\sqrt{\cos\alpha+3}}+3}$
is accurate for an angle $\alpha$ between $0^\circ$ and $120^\circ$
In fact, for $\alpha = 90^\circ$, the result is exact.
How can we derive it?
On
Close but not exact even with regard to the linear term. With $x$ and $y$ as per Paramanand Singh's answer, if $x=1-\delta$, then, ignoring $o(\delta)$ throughout,
$$\begin{align} y&=(1-\delta)[4(1-2\delta)-3]\\ &=(1-\delta)(4-8\delta-3)\\ &=(1-\delta)(1-8\delta)\\ &=1-9\delta. \end{align}$$ By Paramanand Singh's equation (2), $$\begin{align} x&\approx \frac12\sqrt{\frac{2(1-9\delta)}{\sqrt{1-9\delta+3}}+3}\\ &\approx \frac12\sqrt{\frac{2(1-9\delta)}{\sqrt{4-9\delta}}+3}\\ &\approx \frac12\sqrt{\frac{1-9\delta}{\sqrt{1-9\delta/4}}+3}\\ &\approx \frac12\sqrt{\frac{1-72\delta/8}{1-9\delta/8}+3}\\ &\approx \frac12\sqrt{1-63\delta/8+3}\\ &\approx \frac12\sqrt{4-63\delta/8}\\ &\approx \sqrt{1-63\delta/32}\\ &\approx 1-63\delta/64. \end{align}$$
Let $y = \cos \alpha$ and $x = \cos(\alpha/3)$ then we know that $$y = x(4x^{2} - 3)\tag{1}$$
Your approximation says that $$x \approx \frac{1}{2}\sqrt{\frac{2y}{\sqrt{y + 3}} + 3}\tag{2}$$ or using $(1)$ we get $$\frac{y}{x} = 4x^{2} - 3 \approx \frac{2y}{\sqrt{y + 3}}\tag{3}$$ Canceling $y$ we get $$\frac{1}{x}\approx \frac{2}{\sqrt{y + 3}}\tag{4}$$ or $$4x^{2} \approx y + 3$$ or $$y \approx 4x^{2} - 3$$ Comparing with $(1)$ we see that the above approximation is true if $x = 1$ and is good enough if $x$ is near $1$. So the overall approximation is good enough if $\alpha$ is near $0$. The approximation is also correct if $y = 0$ because in this case the LHS and RHS of $(3)$ are equal to $0$.