I am having a serious issue understanding an example in a book titled, Essentials of Stochastic Process by the author, Durrett,
In particular, I do not understand where the inverse matrix comes from and how that fits into anything.
My understanding of determining a stationary distribution rests on
Definition: of qp=q, then q is a stationary distribution.
Any help is appreciated.
On
I think this question breaks down to several parts:
First of all, name $P$ as the given transition matrix, denote $P_1 = I - P$. Assuming that DTMC is irreducible, then there exists a unique stationary distribution $\pi$ such that $\pi P = \pi$. At the same time, rows of $P_1$ sums up to zero for each row.
This gives us $\pi P_1 = 0$. Remember that "rows of $P_1$ sums up to zero", this means the equations of $\pi P_1 = 0$'s last column is redundant. We can simply replace last column of $P_1$ by ones and since $\pi$ is a distribution row vector, it sums to one. Therefore $\pi P_2 = [0,0,...,0,1]$, where $P_2$ is $P_1$ but last columns replaced by ones.
If we can prove that $P_2$ is non-singular, then we can use $\pi = [0,0,...,0,1]P_2^{-1}$ to solve stationary distribution $\pi$.
Before proceeding to next step, for reducible DTMC this is generally not true. Consider $P = \begin{bmatrix}1 & 0 & 0\\0 & 1 & 0\\0 & 0 & 1\end{bmatrix}$, $P_2 = \begin{bmatrix}0 & 0 & 1\\0 & 0 & 1\\0 & 0 & 1\end{bmatrix}$, which is apparently singular.
But for irreducible DTMC, there exists a unique stationary distribution $\pi$. If $P_2$ is singular, then there are two stationary distributions $\pi_1 P_2 = [0,0,...,0,1]$ and $\pi_2 P_2 = [0,0,...,0,1]$ therefore $\pi_1 P_1 = 0$, $\pi_2 P_1 = 0$, which gives contradiction. So $P_2$ must be non-singular.
I think there are still some problems with the proof. But the big picture should be in the right direction.
Looking at the last row of $A^{-1}$ is equivalent to the matrix product $e_nA^{-1}$, where $e_n$ is the row vector made up of $n-1$ zeroes followed by a single $1$. This, in turn, is equivalent to solving the system of equations $xA = e_n$.
Where did this system of equations come from? It's a way of combining the $n \times n$ system $xP = x$ (where $P$ is the transition matrix) with the condition that all the entries of $x$ must add up to $1$.
We can rearrange $xP = x$ into $x(P-I) = 0$, and write the add-up-to-$1$ condition as $xj = 1$ (where $j$ is a column vector of $n$ ones). If we just combine these equations, we get the system $$ x \begin{bmatrix} P-I & j\end{bmatrix} = e_{n+1}. $$ (Similarly to earlier notation, $e_{n+1}$ is a row vector in $\mathbb R^{n+1}$ with $n$ zeroes followed by a $1$.)
We can't invert this yet, because it's not a square matrix. It's not a square matrix because we've imposed $n+1$ conditions on our $n$ variables. The only reason we actually have any solutions here is that of these $n+1$ conditions, one is redundant. Any one of the $n$ equations coming from $xP = x$ can be deleted, because we can deduce it from the other $n-1$.
So we replace $P-I$ by $(P-I)'$: the matrix we get from deleting the last column of $P-I$ (that last column corresponds to the last of the stationary equations). Then we get the system of equations $$ x \begin{bmatrix}(P-I)'&j\end{bmatrix} = e_n $$ and now we can solve this by taking the matrix inverse: $x = e_n \begin{bmatrix}(P-I)'&j\end{bmatrix}^{-1}$.
So that's where the matrix we're inverting comes from: it's made up of the first $n-1$ columns of $P-I$, followed by a column of all ones.