GIVEN:
- $X_0,X_1,...$ irreducible, recurrent Markov chain with transition matrix $P$
- Starting state $X_0=x$
- $g(m)=P\{X_m=y\}$ for some fixed state $y$
I know that the renewal process is $g(m)=b(m)+\sum\limits_{k=1}^m P(Y_1=k)\cdot g(m-k)$, where $Y_1$ is the size of any jump from a state to state and $b(m)$ is another some sequence.
I have a trouble linking Discrete Markov chain and renewal theory. I know that every state change in markov is renewal theory but it seems hard to answer the following questions.
- how to construct a renew equation for g(m)?
- $lim_{m\rightarrow\infty} P\{X_m=y\}$ - I suppose this can be answered easily by Key Renewal Theorem if I figure out what $g(m)$ is though
- Do we get the same renewal equation if $X_0,X_1,...$ is a regenerative process, instead of MC? If not, what is the renewal equation?