I used inequality between arithmetic and geometric means to show that a sequence $\{a_{n}\}$ is bounded: $$a_{n+2}=\sqrt{a_{n+1}a_{n}}\le \frac{a_{n+1}+a_{n}}{2}$$
Solving this, I get quadratic inequality $$a^2_{n+1}-2a_{n+1}a_{n}+a^2_{n}\ge 0$$
which gets me to $$0\le a_{n}\le 1$$ thus, sequence is bounded.
I get that sequence is not monotonic, because $$a_{n}\le a_{n+2} \le {a_{n+1}}$$ or $$a_{n+1}\le a_{n+2} \le {a_{n}}$$
Is this right?
Let $c_n=\log a_n$. Then the sequence $\{c_n\}_{n\in\mathbb N}$ satisfies the recursive relation: $$ c_{n+2}=\frac{1}{2}c_{n+1}+\frac{1}{2}c_{n}, \quad c_1=\log a,\,\,c_2=\log b. $$ Then we have $$ c_{n+2}-c_{n+1}=-\frac{1}{2}\big(c_{n+1}-c_{n})=\cdots=\frac{(-1)^n}{2^n}(c_2-c_1). \tag{1} $$ Also $$ c_{n+2}+\frac{1}{2}c_{n+1}=c_{n+1}+\frac{1}{2}c_{n}=\cdots=c_{2}+\frac{1}{2}c_{1}. \tag{2} $$ Combining $(1)$ and $(2)$ we obtain $$ c_{n+2}=\frac{1}{3}\left(2c_2+c_1+\frac{(-1)^n}{2^n}(c_2-c_1)\right) $$ and hence $$ a_n=a^{1/3}b^{2/3}\left(\frac{b}{a}\right)^{(-1)^n/2^{n-2}}\to a^{1/3}b^{2/3}, $$ since $c^{1/2^n}\to 1$, for every $c>0$.
Note. Proving convergence, without finding the limit, is simpler, as $\{a_{2n+1}\}$ is increasing, $\{a_{2n}\}$ is decreasing, and $a_{2n+2}/a_{2n+1}=\sqrt{a_{2n}/a_{2n-1}}=(a_2/a_1)^{1/2^n}\to 1$.