The paper I am reading has stated the following (without proof):
"For a non-archimedean field $K$ endowed with a non-archimedean valuation $|\cdot|$, the set:
$$ \mathcal{O} = \{\alpha \in K: |\alpha|\leq 1\} $$
is the ring of integers for $K$."
Is this a definition, or does there exist a formal proof of this fact in the sense that $\mathcal{O}$ can be shown to contain all integral elements of $K$ i.e. same ring of integers in algebraic number theory?
It's a definition.
To make sense of integral elements one needs a base ring. If $K$ is a number field, one can define the ring of integers as the elements integral over $\mathbf{Z}$ (or, more generally, the ring of integers of any subfield). If $K = \mathbf{Q}_p$, then one doesn't a priori have a natural subring to work with. In particular, you would want all elements of $K$ to be algebraic over the base ring, otherwise you don't have any hope of having the fraction field of the integral closure of your base ring being $K$ (this eliminates the obvious choice: $\mathbf{Z}$).
However, if $K$ is a finite extension of $\mathbf{Q}_p$, then you can check that the integral closure of $\mathbf{Z}_p$ in $K$ is the same as the definition given above. More generally, if $L/K$ is a finite extension of non-archimedean fields then $\mathcal{O}_L$ is the integral closure of $\mathcal{O}_K$ in $L$.