How can we find minimum radius of circle which contains $\arctan^2(x)+\arctan^2(y)=a$, where I think $a<\pi/2$?
For example I have some plots from WolframAlpha and I see it depends on $a$.
But have no idea how to find maximum radius because this is not simple implicit function.

Seems like the smallest circle is always tangent at $x=0$ or $y=0$. So you can put $\arctan(x)^2=a$ and solve $x=\tan\sqrt a$. This would be the radius.
Edit: let's make it rigorous.
Let's maximize $x^2+y^2$ with the constraint $\arctan(x)^2+\arctan(y)^2=a$. By symmetry we can work in the positive quadrant. With Lagrange multipliers we find $$ \left(\frac{\arctan(x)}{1+x^2},\frac{\arctan(y)}{1+y^2}\right) = \lambda (x,y), $$ therefore (unless $x=0$ or $y=0$, which are easily seen to be stationary points) we have $$ \frac{\arctan(x)}{(1+x^2)x} = \frac{\arctan(y)}{(1+y^2)y}. $$ We want to show that the only possibility is $x=y$. The we need an argument to say that this is a minimum and not a maximum.
As a matter of fact, the function $\frac{\arctan(x)}{(1+x^2)x}$ is decreasing, so this shows that $x=y$ are the only solutions. Its derivative is in fact $$ \begin{split} \frac{d}{dx}\frac{\arctan(x)}{(1+x^2)x} &= \frac{2}{x \left(x^2+1\right)^2}-\frac{2 \arctan(x)}{x^2 \left(x^2+1\right)}-\frac{4 \arctan(x)}{\left(x^2+1\right)^2} \\ &< \frac{2}{x \left(x^2+1\right)^2}-\frac{2 \arctan(x)}{x^2 \left(x^2+1\right)} \\ &= -2 \frac{(1+x^2)\arctan(x)-x}{(x+x^3)^2} < 0. \end{split} $$
You may rightfully wonder why is $(1+x^2)\arctan(x)-x>0$? Well, it vanishes at $0$ and its derivative is $2x\arctan(x)>0$.
So, why is $x=y$ a minimum? Well, we just need to compare it with the point at $y=0$.
The values of $x^2+y^2$ at the two points are respectively $2(\tan\sqrt{a/2})^2$ and $(\tan\sqrt a)^2$. Since the function $f(t)=(\tan\sqrt t)^2$ is convex and vanishes at $0$, it is superadditive, implying that $2f(a/2)\leq f(a)$.
Bonus
I'd like to add one little piece, related to proving that $(1+x^2)\arctan(x)-x$ is increasing. The insight is that it is of the form $\frac{g(x)}{g'(x)}-x$ with $g$ concave. Then it's derivative is $-\frac{g(x)g''(x)}{g'(x)^2} \geq 0$. I don't know if this general detail might be useful to someone.