According to Wikipedia, the Dirichlet series $$\sum^\infty_{n=1}\frac{a_n}{n^s}$$ converges absolutely for $Re(s)>k+1$ where $a_n$ is $O(n^k)$.
For the reciprocal of Riemann zeta function $\frac1{\zeta(s)}$, $a_n=\mu(n)$. RH conjectures that the series converges for $\Re(s)>\frac12$.
Am I allowed to thus say that RH conjectures $\mu(n)$ is $O(n^{-\frac12})$?
You have misunderstood what Wikipedia says. From the convergence theory of Dirichlet series it follows that if the Mertens function $M(x) = \sum_{n \leq x}\mu(n)$ is $O(x^{1/2 + \epsilon})$ for all $\epsilon > 0$, then RH holds. You are certainly not allowed to say that $\mu(n)$ is $O(n^{-1/2})$, for $\mu(n)$ takes only the values $0,{\pm}1$, so on your say-so it would have only finitely many nonzero values. Then $\zeta(s)$ would have no zeros at all! Which is false. BTW the approach towards RH via $M(x)$ is (probably) due to Stieltjes around 1885.