If $p_i$ is the probability of hitting $i$ when starting at 0 on a simple random walk on the integers, I am struggling to understand thoroughly why $p_i=p_1^i$ due to the Markov property.
I understand heuristically, hitting $i$ is ‘like hitting 1 $i$ times’, but I would love to see a formal derivation/proof of why this is the case.
We can show this by induction: for $i=1$ this result is obvious. So suppose $p_i = p_1^i$ for some $i\ge 1$. We want to show $p_{i+1} = p_1p_i$.
Define a stopping time $\tau$ to be the first time our SRW $X$ hits $i$, formally we let $$\tau = \inf\{n\ge 1: X_n = i\}.$$ Then the event $\{\tau<\infty\}$ is precisely the event that $X$ hits $i$, so $\mathbb{P}(\tau<\infty) = p_i$.
The strong Markov property now tells us that conditional on $\{\tau<\infty\}$, $(X_{n})_{n\ge \tau}$ is independent of $(X_n)_{n\le \tau}$ given $X_\tau$. Therefore conditional on $\{\tau<\infty\}$, $(X_n)_{n\ge \tau}$ is a SRW starting from $X_\tau = i$. In particular if $Y$ is a SRW starting from $i$, $$\mathbb{P}(\exists n\ge \tau : X_n=i+1|\tau<\infty) = \mathbb{P}(\exists n\ge 0: Y_n = i+1) = p_{1}.$$ Since $X$ must hit $i$ first if it is to hit $i+1$ we have $$\begin{align*} p_1&=\mathbb{P}(\exists n\ge \tau : X_n=i+1|\tau<\infty)\\ &= \mathbb{P}(\exists n\ge 0: X_n=i+1|\tau<\infty)\\ & = \frac{\mathbb{P}(\exists n\ge 0: X_n=i+1,\tau <\infty)}{\mathbb{P}(\tau<\infty)}\\ &= \frac{\mathbb{P}(\exists n\ge 0: X_n=i+1)}{\mathbb{P}(\tau<\infty)}\\ &= \frac{p_{i+1}}{p_i}. \end{align*}$$