The differential equation $(D^2 + aD + b)y = 0$ has a bounded solution in $[0, \infty)$ then
a>0, b<0
a>0, b>0
a<0, b>0
a<0, b<0
Mark the correct option.
The auxilliary equation will be $m^2 + am + b = 0$.
And solving this we get:
$$ m = \frac{-a \pm \sqrt{a^2 - 4b}}{2}$$
$$ m = -\frac{a}{2} \pm \frac{\sqrt{a^2 - 4b}}{2}$$
And one solution will be
$$y = e^{\frac{-a}{2}}e^{\pm\sqrt{a^2 - 4b}}$$
How to proceed further?
First of all your expression:
$$y = e^{\frac{-a}{2}}e^{\pm\sqrt{a^2 - 4b}}$$
does not make much sense. Where is $x$, for example? So let's start from the beginning:
Solutions of the characteristic equation are:
$$m_{1,2}=\frac{-a\pm\sqrt{a^2-4b}}{2}$$
Case 1: If both solutions are real and different, the solution is:
$$y=C_1e^{m_1x}+C_2e^{m_2x}\tag{1}$$
Case 2: If both solutions are real and equal ($m_1=m_2=m$), the solution is:
$$y=C_1e^{mx}+C_2 x e^{mx}\tag{2}$$
Case 3: If both solutions are complex ($m_1=\alpha+\beta i, m_2=\alpha-\beta i$), the solution is:
$$y=e^{\alpha x}(C_1 cos\beta x+C_2 sin\beta x)\tag{3}$$
If you want a bounded solution for $x\in[0,\infty)$, you have to make sure that, regardles of the case, the exponent to every $e$ appearing in the solution is always less than or equal to zero. Otherwise $e^{mt}$ goes to infinity as $t$ increases and $y$ is not bounded.
If $a>0, b<0$, that's case (1) and one solution is guaranteed to be positive. So $y$ will not be bounded.
If $a>0, b>0$, that's either case (1) with both solutions real and negative ($a^2>4b$), or case (2) with both solutions real, negative and equal ($a^2=4b$), or case (3) with both solutions complex with the real part being negative ($a^2<4b$). In all these cases, solution contains $e$ to the negative power of $x$ and the solution is bounded.
If $a<0, b>0$, that's either case (1) with one real solution guaranteed to be positive ($a^2>4b$), or case (2) with both solutions real, positive and equal ($a^2=4b$), or case (3) with both solutions complex with the real part being positive ($a^2<4b$). In all these cases, solution contains $e$ to the positive power of $x$ and the solution is not bounded.
If $a<0, b<0$, that's case (1) and you get two real solutions with one real solution guaranteed to be positive. So $y$ will not be bounded.