The following problem is from the book Stochastic Processes An Introduction Third Edition by Peter W. Jones & Peter Smith
The picture shows a maze with entrance $E_1$ and further gates $E_2$, $E_3$, $E_4$, $E_5$, and $E_6$ with target $E_7$. These gates can be represented by states in a Markov chain. At each $E_1 , \dots , E_6$ there are two possible new paths which are assumed equally likely to be chosen. The target $E_7$ can be considered to be an absorbing state.
1) Construct a $7 \times 7$ transition matrix for the maze assuming that the walker does not return to a previous state and does not learn from previous choices: for example, at E1 he or she can still make the mistake of walking the dead-end again.
2) Find the probabilities that the walker reaches the target in $6,7,8,\dots$ steps.
3) Suppose now that the walker learns from wrong choices. To accommodate this, let $E_{11}$ be the entrance and $E_{12}$ the return to the entrance after a wrong choice (the dead-end); let $E_{21}$ and $E_{22}$ have the same meaning for the second entrance, and so on. Hence the probabilities are:
$\mathbb{P}(E_{12} \to E_{12})= 1/2$
$\mathbb{P}(E_{11} \to E_{21})= 1/2$
$\mathbb{P}(E_{12} \to E_{21})=1$
$\mathbb{P}(E_{21} \to E_{22})= 1/2$
$\mathbb{P}(E_{31} \to E_{21})= 1/2$
$\mathbb{P}(E_{22} \to E_{31})=1;$
and similarly for the remaining probabilities.
The transition matrix is now $13 \times 13$, with states $E_{11}, E_{12}, E_{21}, E_{22},\dots, E_{61}, E_{62}, E_{7}.$ Find the probabilities that the walker reaches the centre in $6,7,8,\dots$ steps.
My effort
For 1) I have designed the following first order transition probabilities matrix. To my understanding the walker starting at $E_{1}$ can either go to $E_{2}$ or not enter in the maze (i.e stay at $E_{1}$).All the other states from $E_2$ up to the state $E_6$ the probabilities are 1 which means that the walker continue to the next state (path) until reaches the state $E_7$ which is an absorbing state (i.e if the walker enters in state $E_{7}$ stays there forever, and therefore this state has probability also 1.)
$$P = \begin{bmatrix} 1/2 & 1/2 & 0 &0 &0&0&0\\ 0&0&1&0&0&0&0\\ 0& 0&0&1&0&0&0\\ 0&0&0&0&1&0&0\\ 0&0&0&0&0&1&0\\ 0&0&0&0&0&0&1\\ 0&0&0&0&0&0&1 \end{bmatrix}$$
About 2) now the initial distribution of the chain is: $$\pi^{0} = [1,0,0,0,0,0,0]$$ because I enter from state $E_{1}$. Now I have to calculate the $$\pi^{0}P^{6}=\bar{\pi}$$ and from this vector to assess the $\bar{\pi_{7}}$ right?
about the 3) I would like some help because I cannot understand the pattern how the writers describe the transition prrobabilities.
Any help ?