Let $A$ be a stochastic matrix. Prove that $$\lim_{k\to\infty}\frac{1}{k}\sum^{k-1}_{i=0}A^i$$ exists.
This is my attempt.
Let
$$T^{-1}AT = \begin{bmatrix} A_1 & 0 & 0 \\ 0 & \ddots & 0 \\ 0 & 0 & A_m \\ \end{bmatrix} $$
Then,
$$ T^{-1} \left( \frac{1}{k}\sum^{k-1}_{i=0}A^i \right)T = \frac{1}{k}\sum^{k-1}_{i=0}(T^{-1}AT)^i =\begin{bmatrix} B_1 & 0 & 0 \\ 0 & \ddots & 0 \\ 0 & 0 & B_m \\ \end{bmatrix} $$
where $A_1, \dots, A_m, B_1, \dots, B_m$ are Jordan blocks, and $$B_j = \frac{1}{k}\sum^{k-1}_{i=0}A_j^i$$ Since $A$ is a stochastic matrix, then its maximum eigenvalue in magnitude is at most $1$. Then, the diagonal entries of $B_j$ converge to $0$ as $k \to \infty$, whenever the eigenvalue of that block matrix is not $1$, and converges to $1$ when the eigenvalue is $1$. How can I prove that the elements that are not in the diagonal of $B_j$ also converge?