I consider the inequality as follows: let $a>0,b>0$, and satisfy $$2a^2-b^2\leq C(1+a).\tag{1}$$ If we assume that $b\leq a$, then there exist a constant $C_1>0$ such that $$a\leq C_1\tag{2}$$ by solving the quadratic inequality $$a^2-Ca-C\leq 0.\tag{3}$$ where $C_1:=\frac{C+\sqrt{C^2+4C}}{2}.$
Now if we instead of $b \le a$ just assume that $b\leq C_1$, can we conclude that also $$a\leq C_1.\tag{4}$$ holds? (Actually, I wish $(4)$ is true.)
Any answer will be appreciated!
Such a constant does not exist. Take $b$ very close to 0, and we know that $ \frac {2a^2} {1+a}$ is unbounded. Hence, given any $C$, we can find an $a$ (large enough) and a $b$ (close to 0) such that $2a^2 - b^2 > C (1+a)$.
If $a$ is fixed, then you can simply take $C_a = \frac {2a^2}{1+a}$.