Today I was trying to do this exercise:
Find for which $\lambda \in \mathbb{R}$ $$x+x^2=\arctan(\lambda x+x^2)$$ has exactly one solution.
My attempt: Let's define $f(x)=x+x^2-\arctan(\lambda x + x^2)$. Finding solutions of that equation is equivalent to find zeros of this function. We note that $f(x)$ is bounded below by some constant $-M$ as its limits are both $+\infty$ as $x \to \pm \infty$ and it is a continuous function.
Let's try to study its derivative: $$f'(x)=\frac{2x^5+(4\lambda+1)x^4+(2\lambda+2\lambda^2)x^3+\lambda^2x^2+1-\lambda}{1+(\lambda x+x^2)^2}$$
so clearly its sign depends only on the numerator which is a polynomial of degree $5$ in $x$. Before going on we should notice that there are mainly 4 cases to consider:
$$\lambda>1, \quad \lambda=1, \quad \lambda=0, \quad \lambda < 1 \wedge \lambda \neq 0;$$
The problem is that I don't really know how to study the sign of the derivative as it's difficult to study the numerator sign. I tried to avoid studying the sign of the numerator by noting that if we approach the problem in another way, comparing the two functions $g(x)=x+x^2$ and $h(x)=\arctan(\lambda x+x^2)$ we notice that $h(x)$ has only one global minimum in $x= -\frac{\lambda}{2}$ and $g(0)=h(0)=0$ so clearly we should pay attention (expecially in the case $\lambda=1$) on which function is bigger than the other one. Now I'm stuck and I don't know how to proceed anymore. Any hint or help is really appreciated.
Good question and exploration. Your direction is correct. Let $f(x)=x+x^2-\arctan(\lambda x + x^2)$, then the derivative is
$$f'(x)=\frac{2x^5+(4\lambda+1)x^4+(2\lambda+2\lambda^2)x^3+\lambda^2x^2+1-\lambda}{1+(\lambda x+x^2)^2}$$
The key point is to study the behavior of $f(x)$ and $f'(x)$ for small $x$ and large $x$.
At $x = 0$, $f(x) = 0$. As $x\rightarrow \pm \infty$, $f(x)\rightarrow +\infty$.
For small $x$, the numerator of this derivative is $g(x) = O(x^2)+1-\lambda$. For large $x$, the numerator is $g(x) = O(x^5)$.
Therefore, if $\lambda>1$, then $g(x)<0$ for some $\delta>0$ and $|x|<\delta$, which implies that $f(x)<f(0)=0$ for $x\in (0,\delta)$. But we know that finally $f(x)\rightarrow+\infty$ as $x\rightarrow +\infty$, so by the continuity of $f(x)$, there exists a point $x_0\in [\delta,+\infty)$ such that $f(x_0) = 0$. So $\lambda\leq 1$.
Similarly, if $\lambda<1$, then $g(x)>0$ for some $\delta>0$ and $|x|<\delta$, which implies that $f(x)<f(0)=0$ for $x\in (-\delta,0)$. But we know that finally $f(x)\rightarrow+\infty$ as $x\rightarrow -\infty$, so by the continuity of $f(x)$, there exists a point $x_0\in (-\infty,-\delta]$ such that $f(x_0) = 0$. So $\lambda\geq 1$.
In conclusion, we have $\lambda = 1$, and only in this case can $f(x)$ have the single zero $x=0$.