From Haykin's Adaptive filter theory:
A stochastic process is an infinite number of realizations of a process:
A time series is defined as a particular realization of a stochastic process
He also has this:
Consider a discrete-time stochastic process represented by the time series $u(n), u(n - 1), . . . , u(n - M)$, which may be complex valued. To simplify the terminology, we use $u(n)$ to denote such a process. We define the mean-value function of the processs $\mu(n) = E[u(n)]$ where $E$ denotes the statistical expectation operator.
We define the autocorrelation function of the process $r(n, n - k) = E[u(n)u(n - k)], k = 0, {1, {2, . . . }}$
If you have some time series of say $1000$ data points $u(n), \dots, u(n-1000)$, how do you take the expected value and autocorrelation function of this time series? $E(X) = \sum_{x \in X} x P_X(x)$ is the expected value of a random variable, but since each point is a single value how can I find either of these functions?
Note that your mean depends on the sample, i.e. this is not a stationary process, so $$\mu(k) = u(n-k) \qquad k = 0 \ldots 1000$$ The same applies to the autocorrelation function $$r(i,j)=E(u(i)u(j)] \qquad 0 \leq i \leq 1000 ;\ 0 \leq j \leq 1000$$ So your question should be how do we estimate $\mu(k)$ and $r(i,j)$. On way of doing so is to estimate $\mu(k)$ as a sample mean $$\hat{\mu}(k) = u(n - k) \qquad k = 0 \ldots 1000$$ and $$\hat{r}(i,j)=u(i)u(j) \qquad 0 \leq i \leq 1000 ;\ 0 \leq j \leq 1000$$
PS: Everything is completely different when you have a stationary process.