An increasing integrable process $A_t$ is natural if $E\int_0^t m_s dA_s = E\int_0^t m_{s-} dA_s$ for every bounded right-continuous martingale.
If both the Reimann-Stieltjes integrals $\int_0^t m_s dA_s, \int_0^t m_{s-} dA_s$ exist, then I think it can be shown that they should be equal almost surely. Therefore, the definition of a natural process seems 'un-natural', as it requires only equality of expectations. I am not sure what I am missing.
Not at all. Let $A_t = \mathbf{1}_{[1,+\infty)}(t)$ and $m$ be a martingale that always jumps at $1$, e.g. $m_t = \xi \mathbf{1}_{[1,+\infty)}$, where $\xi=\pm 1$ with probability $1/2$. Then $0=\int_0^2 m_{s-} dA_s \neq \int_0^2 m_{s} dA_s = \xi$.
However, this process $A$ is natural (what can be more "natural"?) e.g. $\mathbb{E}\int_0^t (m_s-m_{s-}) dA_s = \mathbb{E}\xi = 0$.
Another useful fact: a process is natural iff it is predictable.