I have a markov chain with transition probabilities $p_{00} = p_{11} = p_{22} = 1-p$ and $p_{01} = p_{12} = p_{20} = p$ for some p between 0 and 1. $f_{ij}^n$ is defined as the probability that the chain enters state j in exactly n steps and starting in state i.
Now I have 3 questions:
- Find $f_{10}^n$.
I tried creating the following system of equations, but I get as a result $f_0=f_1=f_2$
\begin{cases} f_0=pf_1+(1-p)f_0 \\ f_1=pf_2+(1-p)f_1 \\ f_2=pf_0+(1-p)f_2 \end{cases}
- I know that $F_{12}(s) = ps + (1-p)sF_{12}(s)$ because starting in state 1 we are in state 2 with probability p and in state 1 with probability 1-p (so we need to repeat the process $F_{12}$). How can I expess this in a more mathematical way?
here $F_{ij}(s)$ is equal to $\sum s^nf_{ij}^n$.
- Find the expected first pasagge time to state 0 given we start at 0. Now, here I know that if they were asking the first passage time to state 1 I would solve a system of equations and I would set the time to state 0 equal to 0. But how can I solve this know? I was considering using probability generating functions but I do not know how to use them exactly.