how to find the ladder operators for this hamiltonian: $$\widehat{H}=a\widehat{A}^2 + b\widehat{B}^2$$ where $a$ and $b$ are two real and positive constants.
And how to write the hamiltonian in function of the two ladder operators?
Actually the answer is:$$a_-=\frac{1}{\sqrt{2a}}\widehat{A}+i\frac{1}{\sqrt{2b}}\widehat{B}$$ and $$a_+=\frac{1}{\sqrt{2a}}\widehat{A}-i\frac{1}{\sqrt{2b}}\widehat{B}$$
And the condition on the commutator $\widehat{A}$ and $\widehat{B}$ for having: $[a_-,a_+]=\widehat{1}$, I found: $$[\widehat{A},\widehat{B}]=i\sqrt{ab}$$
but I couldn't reach them.
Thank you in advance.
Moving this to an answer as in is too long for a comment: Given a generic Hamiltonian $a\widehat{A}^2 + b\widehat{B}^2$ is not enough information to construct ladder operators. There is no reason to expect that the commutation relation of two conjugate linear combinations of $A,B$ is 1. So you need to be supplied with either the commutation relations of $[A,B]$ or the explicit construction of $A,B$ in terms if $a_{+},a_{-}$.
In the question you say that
$a_-=\frac{1}{\sqrt{2a}}\widehat{A}+i\frac{1}{\sqrt{2b}}\widehat{B}$
$a_+=\frac{1}{\sqrt{2a}}\widehat{A}-i\frac{1}{\sqrt{2b}}\widehat{B}$
So you can plug back in and get the Hamiltonian in terms of $a_{+}, a_{-} $ but the Hamiltonian will now contain mixed terms: $\alpha a_{+}^2+\beta a_{-}^2+\gamma a_{+} a_{-}+\delta a_{-} a_{+}$ for some parameters $\alpha,\beta,\gamma,\delta$.
on the other hand, if you are given $[\widehat{A},\widehat{B}]=i\sqrt{ab}$ , then the construction of a pair of conjugate operators that have a commutation of $1$ is also direct.