Consider a $4$ state Markov Chain that includes the row-stochastic transition matrix $$P = \left(\begin{array}{cccc} 1/4&1/4&1/4&1/4 \\ 1/2&0&0&1/2 \\ 1/3&1/3&1/3&0 \\ 0&0&0&1 \end{array} \right).$$
Assume that we start the chain in state $1$. I would like to find the expected length of time intervals until we are absorbed (as in, before we reach state 4). To me, this seems like a problem where we have a large number of potential transitions, and I need some help on considering a clean way to calculate the terms for the probability that the expected length is $\geq 3$. I see that for the first term of this expectation, we have $1 \times P_{1,4} = \frac{1}{4},$ and for the second term, we have $$2 \times \sum_{i=1}^3 P_{1,i}P_{i,4} = 2 \times (\frac{1}{4} \times \frac{1}{4} + \frac{1}{4} \times \frac{1}{2} + \frac{1}{4} \times 0) = 2 \times (\frac{3}{16}) = \frac{3}{8}.$$
However, I see how this computation becomes much more complicated given the length of the chain. Is there potentially a more efficient way to calculate the individual terms of this expectation?
You don’t really have to compute each path length probability to solve this. Let $v_i$ be the expected number of steps to first reach state 4, the absorbing state, having started in state $i\ne4$. This number includes the first step plus the expected number of steps until absorption starting from whatever state was entered. We thus get the system of linear equations $$v_i=1+\sum_{i\ne4}P_{ij}v_j\tag{1}$$ for $i\ne4$. If we set $v_4=0$ and define $\mathbf r$ as $r_i=1$ for $i\ne4$ and $r_4=0$, this can be expressed in matrix form as $\mathbf v=\mathbf r+P\mathbf v$ or, equivalently, $$(I-P)\mathbf v=\mathbf r.\tag{2}$$ Solving this system yields the expected absorption times. If we omit the all-zero last row and the irrelevant $v_4$ column, this becomes $(I-Q)\mathbf v=\mathbf 1$, with $Q$ defined as in the wikipedia page cited in msm’s answer, and so $\mathbf v=(I-Q)^{-1}\mathbf 1$. (I’ll leave a proof that $I-Q$ is nonsingular for another time.)
This system can, of course be solved via standard row-reduction: $$\left[\begin{array}{ccc|c}\frac34&-\frac14&-\frac14&1\\-\frac12&1&0&1\\-\frac13&\frac13&\frac23&1\end{array}\right]$$ reduces to $$\left[\begin{array}{ccc|c}1&0&0&4\\0&1&0&3\\0&0&1&5\end{array}\right],$$ i.e., $v_1=4$, $v_2=3$ and $v_3=5$.
This can instead be solved via an infinite sum. For $k\ge0$ let $A_k$ be the event “not absorbed after $k$ steps” with $I_k$ the corresponding indicator variable. Then, with a big handwave regarding convergence, the expected time until absorption is $$E\left[\sum_{k=0}^\infty I_k\right] = \sum_{k=0}^\infty E[I_k] = \sum_{k=0}^\infty \operatorname{Pr}(A_k).$$ Recall that $P^k$ gives the $k$-step transition probabilities, so $P^k\mathbf r$, with $\mathbf r$ defined as above, is a vector that gives us $\operatorname{Pr}(A_k)$ for all starting states. Thus $\sum_{k=0}^\infty P^k\mathbf r$ is a vector that consists of all of the expected absorption times. We can again discard the last row and column of $P$ (why?), leaving $\sum_{k=0}^\infty Q^k\mathbf 1=(I-Q)^{-1}\mathbf 1$ as before.