Suppose $(X_n)_{n \geq 0}$ is a discrete-time time-homogeneous Markov chain with transition probabilities $$P[X_{n+1}= y \mid X_{1}=x] = p_{x,y}^{(n)}.$$ Let $$N_x:=\sum_{n \geq 1} 1( X_n=x)$$ denote the number of visits to $x$. We use subscript $x$ to indicate that we start at state $x$.
Why is it true that $$E_x[N_x]=\sum_{n \geq 1} p_{x,x}^{(n)}?$$
At least, often, for defining recurrence, the following is used:
We have that a state is recurrent iff $E_x[N_x]=\infty$ and transient if $E_x[N_x]< \infty$. And often, it is written that a state is recurrent iff $\sum_{n \geq 1} p_{x,x}^{(n)}= \infty$ and transient if the sum is convergent.
I see that $N_x$ takes values in $\{0,1 ,\dotsc\}$, and hence
$$E_x[N_x]=\sum_{n \geq 1} P_x[N_x \geq n],$$ but I do not see why (or whether?) $p_{x,x}^{(n)}$ and $P_x[N_x \geq n]$ are related.
Thank you very much for your help.
$$\mathbb{E}_x N_x=\mathbb{E}_x\sum_{n \geq 1} 1\{X_n=x\}=\sum_{n \geq 1} \mathbb{E}_x[1\{X_n=x\}]=\sum_{n \geq 1} P_x\{X_n=x\}=\sum_{n \geq 1} p^{(n)}(x,x)$$