We have the following situation
The goal is to find $$\lim_{a \to b } \frac{a-b}{c-d} $$
Thought
As $a $ tends to $b$ then we see we are gonna have a rectangle which means that $2 \alpha $ is gonna tend to $ \frac{\pi}{2}$. In other words, $\alpha $ is gonna tend to $\frac{ \pi }{4} $. Now notice
$$ a = \frac{ x }{ \tan \alpha }, b = \frac{ y }{ \tan \alpha} $$
and
$$ d = \frac{ y}{\cos \alpha}, c = \frac{x}{\sin \alpha } $$
now
$$ \lim_{a-b } \frac{ a-b}{c-d} = \lim_{\alpha \to \frac{\pi}{4} } = \frac{\frac{ x }{ \tan \alpha }- \frac{ y }{ \tan \alpha}}{\frac{ y}{\cos \alpha}- \frac{x}{\sin \alpha }} = \frac{x-y}{\frac{2}{\sqrt{2}}x - \frac{2}{\sqrt{2}} y } = \frac{ \sqrt{2} }{2} $$
Is this a correct solution? My

It helps to first spell out what the construction is:
start with the orthogonal "axes" through $O$ and pick point $A$ on the vertical axis;
pick an angle $\alpha \lt \pi/4$ and draw $AB$ such that $B$ is on the horizontal axis and $\angle BAO = \alpha$;
the (other) line through $A$ at angle $\alpha$ with $AB$ intersects the perpendicular in $B$ on $AB$ at $C$;
the vertical through $C$ intersects the horizontal axis at $D$.
It follows from the above that the entire construction is uniquely defined by $a$ and $\alpha$, so the next step is to express the relevant quantities in terms of $a$ and $\alpha$ alone:
$c = \dfrac{a}{\cos \alpha}\;\;(1)\;$ from right triangle $\triangle OBA$;
$d = c \tan \alpha = \dfrac{a \sin \alpha}{\cos^2 \alpha} \;\;(2)\;$ from right triangle $\triangle BCA$ and $(1)$;
$b = d \sin \alpha = \dfrac{a \sin^2 \alpha}{\cos^2 \alpha} = a \tan^2 \alpha \;\;(3)\;$ from right triangle $\triangle DCB$ and $(2)$;
Therefore:
$$\require{cancel} \frac{a-b}{c-d} = \frac{\cancel{a} \left(1 - \tan^2 \alpha\right)}{\dfrac{\cancel{a}}{\cos \alpha}\left(1 - \tan \alpha\right)} = \frac{\cos \alpha\,\cancel{(1-\tan \alpha)}(1+\tan \alpha)}{\cancel{1 - \tan \alpha}} = \sin \alpha + \cos \alpha $$
As the OP well noted $\,a \to b\,$ corresponds to $\,\alpha \to \dfrac{\pi}{4}\,$, so the limit is $\,\sin \dfrac{\pi}{4} + \cos \dfrac{\pi}{4} = \sqrt{2}\,$.
What went wrong in OP's proof started here...
The second part should rather be $\,\color{red}{b = y \cdot \tan \alpha}\,$.
Using $(3)$ above, in fact $\,y=\dfrac{b}{\tan \alpha}=\dfrac{a \tan^2 \alpha}{\tan \alpha}=a \tan \alpha = x\,$.