We know from PDE that a harmonic function $f$ satisfies the mean value property, namely, $f(x)$ = $\frac{1}{\vert{B_r(x)}\vert}\int_{B_r(x)}f(y)dy$ where $B_r(x)$ is the ball about $x$ with radius $r$.
In Markov chain analysis, we also encounter an alternate definition of a harmonic function $f:\mathbb{S}\rightarrow\mathbb{R}$ such that $f(x)$ = $Pf(x)$ := $\sum_{y\in\mathbb{S}}p(x,y)f(y)$, where $P$ is the transition operator of the irreducible Markov Chain, $p(x,y)$ is the one-step transition probability for the chain to travel from $x$ to $y$, and $\mathbb{S}$ is the countable state space.
The clear difference between the two cases is that $f$ in the former one is taken to have $C^{2}$-derivatives and thus a continuous function, whereas in the latter case it is a real function on a discrete domain.
We observe that $Pf(x)$ is indeed $\mathbb{E}f(Y)$ where $Y$ is a Markov Chain started at $x$ with respect to the transition probability measure $p(x,y)$. Taking expectation is like taking the average.
I want to understand if the first definition can be somewhat used to induce the second definition. More precisely, how does the mean value property (now in the discrete case) imply the $f(x)=Pf(x)$ definition.
My attempt:
Since we have not defined any metric on the state space, there is no reason to talk about "neighbourhood" or "center". However, if we infer from the observation that $Pf(x)=\mathbb{E}f(Y)$, $x$ is used here as a starting point. So we might as well take $x$ as a "center" of the neighbourhood $A$, where $A$ := $\{y\in\mathbb{S}:p(x,y)>0\}$. Now, it is hard to quantify the Lebesgue measure analog of $A$. I don't know how to proceed from here.
Do you guys mind shedding some light on this?