A normal to the hyperbola $\frac{x^2}{4}-\frac{y^2}{1}=1$ had equal intercepts on positive x and y axis. If the normal touches the ellipse $\frac{x^2}{a^2}+\frac{y^2}{b^2}=1$, then find $a^2+b^2$
Using equation for normal of hyperbola
$$m=-1$$
So $$y=-x\pm \frac{-(a^2+b^2)}{\sqrt {a^2-b^2}}$$
$$y=-x\pm \frac{-5}{\sqrt 3}$$
Comparing with equation of tangent to ellipse
$$y=-x\pm \sqrt {a^2+b^2}$$ So $$\pm \sqrt {a^2+b^2}=\pm \frac{-5}{\sqrt 3}$$
Which doesn’t seem valid. Even if ignore the sign jargon, and square directly, we end up with $a^2+b^2=\frac{25}{3}$, but the answer is $\frac{25}{9}$
Using a different method, I got the same solution as you did, so it’s likely that the answer key is wrong. That’s been known to happen from time to time.
If the line has equal axis intercepts, then its direction vector is $(-1,1)$. For it to be normal to the hyperbola at some point, by computing the gradient of $x^2/4-y^2$ we have the condition $$\begin{vmatrix}\frac x2&-2y\\-1&1\end{vmatrix} = \frac x2-2y = 0.$$ Solving the system yields $(x,y)=\left(\pm\frac4{\sqrt3},\pm\frac1{\sqrt3}\right)$, so the two normals are $x+y=\pm\frac5{\sqrt3}$. These lines are at a distance of $\frac5{\sqrt6}$ from the origin, so considering the origin-centered circle tangent to the lines, we must have $a^2+b^2=2r^2=2\left(\frac5{\sqrt6}\right)^2 = \frac{25}3$. We can also obtain this value directly from the dual conic: all tangents $lx+my+n=0$ to the ellipse ${x^2\over a^2}+{y^2\over b^2}=1$ satisfy the dual equation $a^2l^2+b^2m^2=n^2$, which here gives $a^2+b^2=\frac{25}3$.
Since $3^2=9$, I suspect that whoever put together the answer key either squared something that shouldn’t have been squared, omitted a square root, or something along those lines.