A scheme is a ringed space locally isomorphic to an affine scheme. An affine scheme is the spectrum of a commutative unital ring together with a structure sheaf.
What are examples of "interesting" affine schemes wich are not (finitely generated) $k$-algebras? Put differently, the definition of a scheme seems very reasonable, but why not take an affine scheme to be something more handy?
I'd be most interested in operations one can perform on schemes modelled on spectra of (f.g.) $k$-algebras whose result is a scheme not modelled on a (f.g.) $k$-algebra. In particular something starting with finitely generated $k$-algebras (truely geometric objects) and going to something which isn't even modelled on spectra of $k$-algebras would be cool.
A number theoretic example would only be "interesting" with an explanation why it is advantageous to consider it in the context of schemes. An advantage would be, for example, a scheme theoretic theorem which may be applied to both, the number theoretic and the geometric context, and yields non-trivial results in both cases.
This question is a little broad. There are a huge number of theorems about schemes, and number theorists use them all the time. Open any of Mazur's papers from the 70's, any of Ribet's papers, any of Gross's papers from the 80's, ... .
One example that I find interesting is that the theory of integral models of curves over a number field is parallel to the theory of surfaces over a field (because a curve over Spec $\mathcal O_K$ is two-dimensional as a scheme), and one has the theory of minimal models, Castelnuovo's criterion, etc., as tools for analyzing the integral models, in complete parallel to the theory of surfaces.
Regarding constructions that pass from finite-type $k$-schemes to more general schemes, note that there are many results saying that various processes preserve being finite-type, etc., so it is not completely trivial to escape the world of finite-type $k$-schemes if that is where you start.
One way is as follows: given $X \to S$ a morphism of finite-type $k$-schemes, and $s \in S$ a point, one can form the base-change $X' \to $ Spec $\mathcal O_{s,S}$ over the canonical morphism Spec $\mathcal O_{s,S} \to S$, where $\mathcal O_{s,S}$ is the local ring of $S$ at $s$.
The base-change $X'$ is finite-type over the local ring $\mathcal O_{s,S}$, but is typically not finite-type over $k$.
One can also make the analogous construction, but with the completed local ring $\widehat{\mathcal O_{s,S}}$.
This is a standard manner by which one leaves the world of finite-type schemes over a field. It is convenient for studying properties of the original morphism locally around the point $s$. Note that Spec of a local ring is typically not Jacobson, so from a technical point of view, this puts one in a scheme-theoretic context which is pretty far from that of finite-type schemes over a field.