From "Elementary Methods in Number theory" by 'Melvyn B. Nathanson',
I know from Merten's first theorem that $\displaystyle R(x)=\sum_{p\leq x}\dfrac{\ln p}p-\ln x=O(1)$ but can it be $O\left(\dfrac{\ln^4x}{x^{1.5}}\right)$ at least for large numbers?
because the thing is, I showed, $$\pi(x)-\operatorname{li}(x)=\sum_{n\leq x}a_n\operatorname{li}(n)-\sum_{n\leq x}a_n\operatorname{li}(n)\left(\operatorname{li}(n)-\dfrac n{\ln n}\right)$$ where $\operatorname{li}(x)$ is the logarithmic integral ($\operatorname{Li}(x)+1.04$) and $\sum_{n\leq x}a_n=R(x)$
$\therefore a_n=O\left(\frac{\ln^4x}{x^{1.5}}\right)$ only if we consider $R(x)=\left(\tfrac{\ln^4x}{x^{1.5}}\right)$, which is my question
(and also the Riemann's hypothesis).
Obviously not. Your asymptotics tend to $0$ for large $x$, whereas that $O(1)$ term in Merten's theorem is $\geq 1$. Precisely,
$$\lim_{x \to \infty} \left | \sum_{p \leq x} \frac{\log p}{p} - \log(x) \right | \approx 1.3325$$
This is usually denoted as $B_3$.
As a final note, your formulation of Merten's theorem is incorrect, and presumably you meant to take the absolute value at the difference.