I am trying to maximize the function $A(x,y)=\frac{1}{2}(x(12-x)+y(13-y))$ subject to the constraint $x^2+(12-x)^2-y^2-(13-y)^2=0$.
My attempt:
$\begin{align*} \nabla A=\frac{1}{2}\langle 12-2x,\,13-2y\rangle &= \lambda\langle4x-24,\, -4y+26\rangle\\ \implies&\begin{cases} -x+6=\lambda(4x-24)\\-y+\frac{13}{2}=\lambda(-4y+26)\\x^2+(12-x)^2-y^2-(13-y)^2=0\end{cases}\end{align*}$
But clearly there is no solution due to the first two equations.
Using Wolfram Alpha, however, yields a maximum at $\displaystyle \left(\frac{17}{2},\,\frac{13}{2}\right)$ being $A=36$ and shows a nice little graph.
First, let's re-explain why the first two equations are contradictory. The first equation gives us:
$$\lambda=\frac{-x+6}{4x-24}=-\frac 1 4$$
While the second equation gives us:
$$\lambda=\frac{-y+\frac{13} 2}{-4y+26}=\frac 1 4$$
However, what's important to realize here is that these two equations only work when $4x-24\neq 0$ and when $-4y+26\neq 0$. Therefore, in order to find a solution, we need to consider the other cases: $4x-24=0$ (i.e. $x=6$) and $-4y+26=0$ (i.e. $y=\frac {13} 2$).
Case 1: $x=6$
Let's plug $x=6$ into our constraint equation: $$6^2+(12-6)^2-y^2-(13-y)^2=0\rightarrow y=\frac{13}{2}\pm \frac{5i}{2}$$ Thus, this equation has no real solutions, and this case can be ignored.
Case 1: $y=\frac {13} 2$
Let's plug $y=\frac {13} 2$ into our constraint equation: $$x^2+(12-x)^2-\left(\frac{13}2\right)^2-\left(13-\left(\frac{13}2\right)\right)^2=0\rightarrow x=\frac 7 2 \text{ or } x=\frac{17}2$$
Thus, we have two critical points: $(\frac 7 2, \frac{13}2)$ and $(\frac{17}2, \frac{13}2)$. I will leave it to you to show these critical points are maximums.