If $0<x<90^\circ$ and $\sin x=24/25$ find the value of $$\frac{\cos(x/2)}{\cos(x/2)+\sin(x/2)}$$
I tried using the half angle tangent substitution,
$$\frac{2\tan(x/2)}{1+\tan^2(x/2)} = \frac{24}{25}$$ then I got the quadratic in terms of $\tan(x/2)$:
$$12\tan^2(x/2) - 25\tan(x/2) + 12=0$$
So, there are 2 roots to this equation, $4/3$ and $3/4$. The answer to the question supports the $4/3$ answer, but I don't know why. Is there a rule picking the highest root?
There are indeed two completely valid solutions for the question.
A much simpler solution can be found by multiplying both the top and bottom by $\sin \frac x2$ (first noting that this value is non-zero), giving:
$$\frac{\cos \frac x2}{\cos \frac x2 + \sin \frac x2} = \frac{\sin \frac x2 \cos \frac x2}{\sin \frac x2\cos \frac x2 + \sin^2 \frac x2} = \frac {\sin x }{\sin x - \cos x + 1}$$
after employing the half-angle formulae.
Since $x$ can lie within any quadrant by the given range (and the positive sine only restricts it to the first two quadrants), $\cos x$ can be positive or negative, so $\cos x = \pm \sqrt{1 - (\frac{24}{25})^2} = \pm \frac{7}{25}$, giving the answer of : $\frac{\frac{24}{25}}{\frac{24}{25} \mp \frac{7}{25}+1} = \frac 47 \ \mathrm{or}\ \frac 37$.
Actually finding $x = \arcsin \frac{24}{25}$ and taking both the admissible first and second quadrant values as possibilities confirms both of these solutions.
EDIT (to address edited question):
If the range of $x$ is restricted to $x \in (0,90^{\circ})$ - i.e. first quadrant, then we're looking for positive $\cos x$, which means that only the value of $\cos x = \frac 7{25}$ is admissible. This makes the original expression $\frac{24}{42} = \frac 47$, and this is the only correct answer.
It is possible to work with the tangent half angle sub too. Note that for $x \in (0,90^{\circ}), \frac x2 \in (0,45^{\circ})$. Because of the strictly increasing nature of the tangent function over a first quadrant domain, that means $\tan \frac x2 < \tan 45^{\circ} = 1$. Hence we accept only the value of $\tan \frac x2 = \frac 34$, which makes the original expression $\frac 1{1+\frac 34} = \frac 47$, just as before.
It should also be noted that the line in the OP's question stating that the answer "supports $\frac 43$" is incorrect.