If $a,b\in R$ and $\displaystyle a^2+b^2=1+\frac{2ab}{a-b}$ and $\sqrt{a-b}=a^2+5b$, what is the maximum of $ab$?
Here is what I tried:
$a=r\cos\alpha$ and $b=r\sin \alpha$
$\displaystyle r^2=1+\frac{r\sin 2\alpha}{\cos \alpha-\sin \alpha}$
and $\sqrt{r(\cos \alpha-\sin \alpha)}=r^2\cos^2\alpha+5r\sin \alpha$
The domain gives $a-b>0$ and from the first equation we obtain: $$(a-b)^2+2ab=1+\frac{2ab}{a-b}$$ or $$(a-b)^2-1+2ab\left(1-\frac{1}{a-b}\right)=0$$ or $$(a-b-1)\left(a-b+1+\frac{2ab}{a-b}\right)=0$$ or $$a=b+1.$$ Can you end it now?
I got the following answer: $42$.