why one would model something (anything?) with a stochastic process, such as Ornstein–Uhlenbeck (OU), rather than a regression, like $y(t) = \beta_0 + \beta_1e^{( - rt)}$? With regression, you can use the standard error to get the probability distribution. So I am not seeing the benefit of stochastic processes.
What is the benefit of OU vs regression for modeling data, say data in the form of (x,y) pairs?
It seems like any stochastic process can be mapped by a regression, maybe equally well. What am I missing?
Stochastic processes and regression analysis are just two sides of the same coin. Namely, Assume that you have a realization from a univariate time process and you postulate that the process that generated this data was autoregression of order $1$, however with an unknown coefficient $\phi$. I.e., $X_t = \phi X_{t-1} + \epsilon_t$, hence you can use statistics (regression analysis) in order to estimate $\phi$. Now, assume that you are not sure what process generated your data and you are willing to test a set of AR(I)MA models. Here too the statistics might help you to select the most appropriate model. Namely, parametric-regression models allow you to approximate the data-generating process by a linear regression. Note that not every stochastic process can be well approximated by a linear-parametric regression model. And, basically, stochastic analysis assume that you know the properties of the process and you can work with them, while regression analysis assume that you don't know the data-generating process and you try to recover its properties by using the data.