If I have Transition matrix
$T=\begin{pmatrix} 2/3 &0 &1/3 \\ 1/4 &3/4 &0 \\ 1/3& 0 &2/3 \end{pmatrix}$
How would I get the quantity $R_i^{(n)}=\sum_{k=1}^{n}(T^k)_{ii}$ I want to plot a graph for $R_i^{(n)}$
for each of the states $i ∈ {1, 2, 3}$ but how would I calculate these different quantities. And is it possible to prove analytically if they are bounded or not.
First, note that for real numbers $x$, we have $$ \sum_{k=1}^n x^k = \begin{cases} \frac{x - x^{n+1}}{1-x} & x \neq 1\\ n & x = 1 \end{cases} $$
Now, with an eigendecomposition, we find that $T = PDP^{-1}$ where $$ D = \pmatrix{1 \\ & 3/4 \\ && 1/3}, \qquad P = \pmatrix{1 & 0 & -5\\ 1 & 1 & 3\\ 1 & 0 & 5}. $$ It follows that $$ \sum_{k=1}^n T^k = P\left[ \sum_{k=1}^n D^k\right]P^{-1} = PMP^{-1} $$ where $M$ is the matrix given by $$ M = \sum_{k=1}^n D^k = \pmatrix{\sum_{k=1}^n 1^k \\ & \sum_{k=1}^n(3/4)^k \\ && \sum_{k=1}^n(1/3)^k} \\= \pmatrix{ n \\ & \frac{3/4 - (3/4)^n}{1/4} \\ && \frac{1/3 - (1/3)^n}{2/3} } \\ = \pmatrix{ n \\ & 3(1 - (3/4)^{n-1}) \\ && \frac 12(1 - (1/3)^{n-1}) }. $$ Once $P$ and $P^{-1}$ are calculated, we have $$ [PMP^{-1}]_{ii} = e_i^TP M P^{-1}e_i = (P^Te_i)^T M (P^{-1}e_i). $$ Note that for any matrix $A$, $Ae_i$ is the $i$th column of $A$. You can now get your formula noting that $$ P^T = \pmatrix{1&1&1\\0&1&0\\-5&3&5}, \qquad P^{-1} = \frac 1{10} \pmatrix{5&0&5\\ -2&10&-8\\ -1&0&1}. $$
For for $i = 1$, use the first columns: $$ R_1^{(n)} = \frac 1{10}\pmatrix{1&0&-5}\pmatrix{ n \\ & 3(1 - (3/4)^{n-1}) \\ && \frac 12(1 - (1/3)^{n-1}) }\pmatrix{5\\-2\\-1} \\= \frac {1\cdot 5}{10}n + \frac {0\cdot(-2)}{10}\cdot 3(1 - (3/4)^{n-1}) + \frac{(-5) \cdot (-1)}{10}\cdot \frac 12(1 - (1/3)^{n-1}) $$