I am having a few problems understanding the Ornstein Uhlenbeck solutions, on wikipedia under solution (http://en.wikipedia.org/wiki/Ornstein%E2%80%93Uhlenbeck_process) it described using variation of parameters with Ito's Lemma but I am struggling in getting df(x,t), I supposed I am also a little stuck in using Ito's lemma, as I am struggling with the definition I found on the wikipedia page.
It goes on to write an alternative form for the analytical solution without an integral, I am having troubles understanding how this equation was arrived at and how it differs from the other solution. Ideally I would look to simulate both solutions computationally afterwards so any advice would be greatly appreciated.
I would also like to get some help in understanding and arriving at, the errors associated with numerical SDEs, for example Milstein and Euler-Maruyama.
Thank you for much for reading this I greatly appreciated it :)
To get the correct function $f$ is just guess-work. Sometimes it helps to treat the stochastic term as inhomogeneity and solve just the "homogenous" part, and then, as mookid said, use variation of constants. Sometimes, this does not work.
So in principle, one could have taken as reduced deterministic equation $$ \dot x=θ(μ-x)\iff e^{θt}(\dot x+θx)=e^{θt}θμ\iff e^{θt}x=μe^{θt}+C $$ to get the idea to explore the quantity $Y_t=(X_t-μ)e^{θt}$, $Y_0=X_0-μ$ and come up with $$ dY_t=dX_te^{θt}+(X_t-μ)θe^{θt}\,dt=σe^{θt}\, dW_t $$
How to get the exponential into the index: The increments $$ Δ W_t=W_{t+Δt}-W_t $$ are random variables with a normal distribution $N(0,Δt)$. The scaled increments $e^{θt}Δ W_t$ are thus normally distributed with $N(0,e^{2θt}Δt)$. Let $\widetilde W_s$ be another Brownian motion, then $$ \widetilde W_{s+e^{2θt}Δt}-\widetilde W_s $$ also has the distribution $N(0,e^{2θt}Δt)$. If we define the connection between $s$ and $t$ as a function $s_t$ with $Δs_t=e^{2θt}Δt$, or in the limit $Δt\to 0$ as $ds_t=e^{2θt}dt$, one gets $s_t=\frac1{2θ}e^{2θt}$. Then $$ e^{θt}\,d W_t\ \text{ and }\ d \widetilde W_{s_t} $$ have always the same distribution, that is, $$ Y_0+σ\widetilde W_{s_t}\ \text{ and }\ Y_t=Y_0+\int_0^te^{θτ}\,d W_τ $$ are equivalent (in distribution) processes.
In the same way that one gets factors into the time, one can also extract them, especially constant factors, so that $$ \tilde Y_t=Y_0+\frac{σ}{\sqrt{2θ}}\widetilde W_{\exp(2θt)} $$ is another process equivalent to $Y_t$, and $$ \tilde X_t=μ+e^{-θt}\tilde Y_t=μ+e^{-θt}\left(X_0-μ+\frac{σ}{\sqrt{2θ}}\widetilde W_{e^{2θt}}\right) $$ is a stochastic process that is equivalent (in distribution) to the solution $X_t$ of the Ornstein-Uhlenbeck equation $$ dX_t = \theta (\mu-X_t)\,dt + \sigma\, dW_t. $$