Show that a random walk on a finite grid graph is recurrent

Question

I want to show that a random walk on a graph is recurrent. The graph is a network of nodes which connect together to make a $N \times M$ rectangular grid, such as this, my first thought was to somehow get an expression for

$\mathbb{P}[\text{on node}\ n\ \text{after}\ i\ \text{jumps}|\text{on node}\ 0\ \text{at}\ i=0]$,

then express

$\mathbb{P}[\text{on node}\ 0\ \text{after}\ i\ \text{jumps}|\text{on node}\ 0\ \text{at}\ i=0]$

then let $i\to\infty$ and hope it $=1$.

Does this sound like the right approach? Any pointers?

Answer 1

You don't need an explicit expression. Any irreducible Markov chain on a finite state space is recurrent.