I was playing around with the definition of a blow up when I encountered something interesting.
Theorem IV-23 Eisenbud & Harris
Let $X$ be a scheme and $Y\subset X$ a closed subscheme. Let $\mathcal J \subset \mathcal O_Y$ be the ideal sheaf of $Y$ in $X$. If $\mathscr A$ is the sheaf of graded $\mathcal O_X$-algebras $$\mathscr A = \bigoplus_{n=0}^\infty \mathcal J^n$$ then the scheme $\operatorname {Proj}\mathscr A \to X$ is the blowup of $X$ along $Y$.
Now taking $X$ to be $\operatorname{Spec} k$ for some algebraically closed field $k$ we should have a variety with a single point. There is only one blow up that can be done, this gives the ideal sheaf as $k$ thus $\mathscr A \cong \mathcal k[t]$. This gives the blow up of $X$ along (essentially) $X$ to be $\operatorname{Proj}k[t]=\Bbb P_k^1$.
I have never heard of a description of protective space over an algebraically closed field $k$ as simply being the blow up of $\operatorname{Spec} k$.
So my questions are as follows:
Is $\Bbb P^1_k$ simply the blow up of $\operatorname{Spec}k$?
Does this generalise to $\Bbb P^1_S$ of some reduced scheme $S$ being the blow up of $S$ at a point? I suppose it isn't quite $\Bbb P^1$ every time...
Is this point of view helpful for understanding the geometry of $\Bbb P^1$?
[Thanks to TomGrubb for correcting some wrong statements that were previously in this answer!]
No: $\operatorname{Proj} k[t]$ is $\mathbb{P}^0_k\cong\operatorname{Spec} k$, not $\mathbb{P}^1_k$. To get $\mathbb{P}^1_k$, you would want a graded polynomial ring in two variables.
Note moreover that $k$ is the ideal sheaf of the empty subscheme of $X=\operatorname{Spec} k$, not of $X$ itself. The ideal sheaf of $X$ would be the zero ideal, and so the graded algebra you get would be the zero algebra whose Proj is empty.
More generally, blowing up a scheme along the empty subscheme never does anything, and blowing up a scheme along the whole scheme gives you the empty scheme. In the case of blowing up along the empty set, the graded algebra $\mathscr{A}$ will just be $\mathcal{O}_X[t]$, and when you take Proj that gives you back just $X$ itself. (In the case that $X=\operatorname{Spec} A$ is affine, $\operatorname{Proj} A[t]$ is covered by the single affine open set you get by inverting $t$, since $t$ generates the irrelevant ideal. Since the degree $0$ part of $A[t,t^{-1}]$ is just $A$, this gives you $\operatorname{Spec} A$.) In the case of blowing up along the whole scheme, $\mathscr{A}$ will just be $0$.