I have read that the infinitesimal generator for a Markov process $(X_t)_{t \geq 0}$ we define the generator $A$ by
$$Af(x) := \lim_{t \downarrow 0} \frac{\mathbb{E}^x(f(X_t))-f(x)}{t} $$
whenever the limit exists in $(C_{\infty},\|\cdot\|_{\infty})$. The definition implies that $$\mathbb E[Af(X)]=0$$ if $X$ is the stationary distribution.
I want to know why the definition is limited to Markov processes and not defined for a general stochastic process?
Just my guess, it's because Markov property makes the infinitesimal generator identify the entire process. This is surprising because the infinitesimal generator is defined only at the neighborhood of $s=0$.
By the definition, the infinitesimal generator is similar to the derivative in calculus, i.e. we want to introduce some derivative-like notion of stochastic process. A straightforward approach to develop it is to define it for all time $s\ge0$. This may be tentatively defined for all $s\ge0$ as: $$ A_sf(x) = \lim_{t\to0} \frac{ \mathbb{E}\left\lbrack f(X_{s+t}) \middle\vert \mathcal{F}_{s} \right\rbrack - f(x)}{t}, $$ where $X_s = x$ and $\mathcal{F}_s$ is the filtration generated by $X_s$.
However, if $\{X_s\}_{s\ge0}$ has the Markov property (and it leads the time-homogeneity), we only need to define it at (the neighborhood of) $s=0$ with the initial value $x$. In fact, for all $s\ge0$, $$ \begin{align} A_s(x) &:= \lim_{t\to0} \frac{ \mathbb{E}\left\lbrack f(X_{s+t}) \middle\vert \mathcal{F}_{s} \right\rbrack - f(x)}{t} \\ &= \lim_{t\to0} \frac{ \mathbb{E}\left\lbrack f(X_{s+t}) \middle\vert X_s = x \right\rbrack - f(x)}{t} \quad (\because \text{Markov property}) \\ &= \lim_{t\to0} \frac{ \mathbb{E}\left\lbrack f(X_t) \middle\vert X_0 = x \right\rbrack - f(x)}{t} \quad (\because \text{time-homogeneity}) \\ &= A_0f(x). \end{align} $$
In other word, if $\{X_s\}_{s\ge0}$ has the Markov property, the derivative-like notion at $t=0$, so called the infinitesimal generator of $\{X_s\}_{s\ge0}$, determines, identifies or generates the entire process with initial value $x$.