Let $\{X_n\}$ and $\{a_n\}$ be sequences of random variables and real numbers, respectively.
Say that $X_n=O_P(a_n)$ iff $\forall\varepsilon>0$, $\exists N,M>0$ such that for all $n>N$, we have $P(\lvert X_n/a_n \rvert>M)\leq\varepsilon$.
In my case, I have $E\lvert X_n \rvert\leq C a_n$, where $C$ is some positive constant and $a_n\to 0$. I want to show that $X_n=O_p(a_n)$. Markov's inequality yields given any $\varepsilon>0$, $$P(\lvert X_n/a_n \rvert>\varepsilon)\leq \frac{E\lvert X_n \rvert}{a_n\varepsilon}\leq \frac{C}{\varepsilon}.$$
My question (probably naive) is that if it is enough to conclude that $X_n=O_p(a_n)$. I'm thinking that if I identify $\varepsilon':=1/\varepsilon$ and note that the range of $\varepsilon'$ on $\mathbb{R}_+$ is $\mathbb{R}_+$, then it implies the result.
Can you clarify me this point?
It is enough. You just need to choose $\varepsilon' = \varepsilon/C$.
Indeed, let $\varepsilon > 0$. Choose $\varepsilon' = \varepsilon/C$ and $M=\varepsilon'$.
Then using $\varepsilon'$ in the inqueality you derived you get, for $n > N$,
$$ P(\lvert X_n/a_n \rvert> M) = P(\lvert X_n/a_n \rvert>\varepsilon')\leq \frac{E\lvert X_n \rvert}{a_n\varepsilon'}\leq \frac{C}{\varepsilon'} = \varepsilon. $$