I was thinking about this sequence
\begin{equation}\tag{1} n=\sqrt{n^2-n+\sqrt{n^2-n+\sqrt{n^2-n+\sqrt{n^2-n+\sqrt{\ldots}}}}} \end{equation} It is well known that this converges to n. For example: $$3=\sqrt{9}=\sqrt{6+3}=\sqrt{6+\sqrt{9}}=\sqrt{6+\sqrt{6+3}}=\sqrt{6+\sqrt{6+\sqrt{6+\ldots}}}$$
The problem is when $n=0$ and $n=1$, because when we apply formula $(1)$, in both cases we get
$$\sqrt{0+\sqrt{0+\sqrt{0+\sqrt{0+\sqrt{\ldots}}}}}$$ This yields $$0=1$$ Can you tell me where the mistake is in the reasoning?
If the nested square root has any meaning at all, it is as the limit of a sequence of finitely nested expressions. You need to specify which ones.
For example, you might specify some initial function $a_0(n)$, and let $a_{j+1}(n) = \sqrt{n^2 - n + a_{j}(n)}$. You then want to prove something about the limit of $a_j(n)$ as $j \to \infty$. It's easy to see that if the limit exists it must be $n$ or $1-n$, these being possible only if they are nonnegative (if we want everything real not allowing negative square roots), because the function $f(t) = \sqrt{n^2-n+t}$ has only those fixed points, but that doesn't say that the limit exists.
Note also that $f'(n) = 1/(2 |n|)$, while $f'(1-n) = 1/(2 |1-n|)$, so the fixed point $n$ is stable if $n > 1/2$ while $1-n$ is stable if $n < 1/2$.