$$\sec\left(\frac{x^2-y^2}{x^2+y^2}\right)=e^a$$
Taking $\sec^{-1}$,
$$\frac{x^2-y^2}{x^2+y^2}=\sec^{-1} (e^a)$$
Now I simply ignore the Right Hand Side as its differentiation is $0$ anyway, so I replace it with $k$ for now. On the other hand (pun not intended), the differentiation of the LHS seems a bit more tricky so I substitute $x$ with $a \cos\theta $ and $y$ with $a \sin\theta $ and then proceed.
$$\frac{a^{2}\cos^{2}\theta -a^{2}\sin^2\theta}{a^{2}\cos^{2}\theta+a^2\sin^2\theta}=k$$
Removing the $a^2$ to make it more friendly, the equation becomes
$$\frac{\cos^{2}\theta -\sin^{2}\theta}{\cos^{2}\theta +\sin^2\theta}=k$$
Using the Trigonometric identities
$$\cos2\theta=k$$
Now differentiating
$$-2\sin2\theta=0$$
$$\sin2\theta=0$$
$$2\sin\theta \cos\theta =0$$
and so
$$2\cdot\frac{x}{a}\cdot\frac{y}{a}=0$$
that should be giving me
$$xy=0$$
whose differentiation is
$$\frac{dy}{dx}=-\frac{y}{x}$$
But the answer given is different.
Your solution has serious drawbacks:
You use the substitution $(x,y)=(a\cos\theta, a\sin\theta)$. (This $a$ conflicts with the $a$ in the problem, but it doesn't matter much.) Now, doing so, you introduce another dependency of $y$ on $x$, which is wrong.
You had $\cos2\theta=k$. Differentiating gives $xy=0$. Okay, but note that this relation between $x$ and $y$ is entirely new, not to do with the original relation.
To see this, consider the relation $x^2-6x+4=k$. Differentiating gives $2x-6=0$ i.e. $x=3$. But does $x=3$ satisfy the original equation? No. Why? Because the differentiated relation is distinct from your original relation.
Similarly, in your case, $\cos2\theta=k$ is satisfied by some $x$ and $y$, but these $x$ and $y$ are not necessarily the same as the $x$ and $y$ you get on differentiating, $xy=0$.
As for your problem, you can simplify the differentiation by using $$\dfrac{x^2-y^2}{x^2+y^2}=\dfrac{2x^2}{x^2+y^2}-1$$
Hope this helps. Ask anything if not clear. :)