Given a recurrence relation: $$ x_{n+1} = x_n^2 + 3x_n + 1 \\ x_1 = a\\ n\in\Bbb N $$ Figure out whether this sequence has a limit (either finite or infinite) and find it for: $$ \begin{align*} a = -{5\over 4}\tag1 \\ a = -{3\over 4}\tag2 \end{align*} $$
Start with case $(1)$. It took some time to notice but seems like the sequence is monotonically increasing no matter what initial conditions are given. That is because: $$ x_{n+1} = x_n^2 + 3x_n + 1 \iff x_{n+1}-x_n = x_n^2 + 2x_n + 1 = (x_n+1)^2>0 $$ Than means: $$ x_{n+1} - x_n > 0 \iff x_{n+1} > x_n $$ That observation is crucial for all the next steps. In $(1)$ we are given that: $$ x_1 = a = -{5\over 4} > -2 $$ By monotonicity of $x_n$: $$ \forall n\in\Bbb N : x_n > -2 $$ Let's suppose the limit exists. Then by finding fixed points of the recurrence we may get an insight of what that limit might be: $$ L = L^2 + 3L + 1 \iff (L+1)^2 = 0 \iff L = -1 $$ Thus the only possible finite limit in $\Bbb R$ is $L=-1$. Let's try to bound $x_n$ above. Using induction: $$ x_1 < x_2 = -{19\over 16} < -1 $$ Suppose $x_n < -1$. Then: $$ x_n \in (-2; -1) \implies \underbrace{(x_n + 1)^2 + x_n}_{x_{n+1}} \in (-2, -1) $$ Thus it follows that $x_{n+1} < -1$. Now by monotone convergence theorem a monotonic bounded sequence has a limit. Therefore: $$ \boxed{\lim_{n\to\infty}x_n = -1} $$
This case is more of a headache. Given $a = -{3\over 4}$ makes the sequence diverge to $+\infty$. But to show this I had to calculate the value for $6$ first terms. It follows that: $$ \forall n \ge 6: x_n > 0 $$ Moreover: $$ \forall n \ge 7: x_n > 1 $$
So: $$ \boxed{\lim_{n\to\infty}x_n = +\infty} $$
Does there exist a more elegant way to solve for case $(2)$?
Also is this argumentation enough to show what's requested in question section? I have doubts about the second case. Because formally I should have shown that the sequence is not bounded, not sure how to do it. And the solution is ugly. Here is a sandbox I've been using to play around with the recurrence.
Could you please verify the above and point to the mistakes just in case? Thank you!
Does there exist a more elegant way to solve for case $(2)$? Yes there is ! You don't need to compute the first six terms at all ! The sequence is nondecreasing, so it either converges to a finite value or diverges to $+\infty$. It if converges, the limit can only be $-1$ as you have shown, but this is impossible since your sequence, being nondecreasing, will always be $\geq x_0=-\frac{3}{4}$.