I need to resolve a gambler's ruin problem so I have been looking into problem number 5 in the following pdf, which is similar to mine and it is resolved:
https://www.webpages.uidaho.edu/~fuchang/StochasticModel/Review3Solution.pdf
Consider the gambler’s ruin problem with p = 0.6 and n = 4. Starting in state 2. Find:
First, they create Q, the matrix that describes the probability of transitioning from some transient state to another. What is a submatrix of the transition matrix
$$ Q=\begin{pmatrix} 0 & .6 & 0 \\ .4 & 0 & .6 \\ 0 & .4 & 0 \\ \end{pmatrix} $$
Then, they calculate the matrix that let us know the expected number of times the chain is in state j, given that the chain started in state i.
$$ (I-Q)^{-1}=\begin{pmatrix} 1.4615 & 1.1538 & 0.6923\\ 0.7692 & 1.9231 & 1.1538\\ 0.3077 & 0.7692 & 1.4615 \\ \end{pmatrix} $$
...
(b) the probability of ever visiting state 1
the probability of ever visiting state 1 is 0.7692/1.4615 = 0.5263
Why is this division supposed to return the probability of ever visiting state 1?
The exercise in the pdf does not have such as section, but I would also need to calculate the expected number of visits to state 3 but in the first 2 transitions. Any hint?
Let $V_1$ denote the total number of visits to state 1. Write $E_i(V_1)$ for the mean of $V_1$ when the chain starts at state $i$. Write $A_1$ for the event that the chain ever reaches state 1. Then using the (strong) Markov property at the time of the first visit to state 1, we obtain $$E_2(V_1)=P_2(A_1) E_2(V_1|A_1)=P_2(A_1)E_1(V_1) \,.$$ That explains why $P_2(A_1)$ is the ratio $E_2(V_1)/E_1(V_1)$.
$----$
For your second question, using additivity of expectation, the expected number of visits to state 3 in the first two transitions (starting at state 2) is $Q(2,3)+Q^2(2,3)$. In the case of the given chain, the second summand vanishes and you obtain simply $Q(2,3)$.