What is the most general transition matrix for a two state Markov Chain? (both Markov and homogeneous). And show that any such chain has an equilibrium vector.
Is it should be the following matrix? If it is, how could we show it alway exists an equilibrium vector? \begin{bmatrix} p & 1-p \\ q & 1-q \end{bmatrix}
You're right, that's the two-state transition matrix in full generality. To find an equilibrium vector, we solve: \begin{align} \begin{bmatrix} x & y \end{bmatrix} \cdot \begin{bmatrix} p & 1-p \\ q & 1-q \end{bmatrix} = \begin{bmatrix} x & y \end{bmatrix}, \end{align} which yields the system \begin{equation} \left\{ \begin{aligned} px+qy&=x \\(1-p)x+(1-q)y&=y \end{aligned} \right. . \end{equation} The second equation is redundant, \begin{align} y &=(1-p)x+(1-q)y \\&=x+y-(px+qy) \\&=x+y-(x) \\&=y, \end{align} so we only have to solve the first one. Now, pick any non-negative value of $x$, then \begin{equation} y=\frac{(1-p)x}{q} \end{equation} is a non-negative number, and the pair $(x,y)$ solve the first equation (check). Finally, if $(x,y)$ solve the first equation, then $(sx,sy)$ for any $s$ also solve the first equation, because \begin{align} sx &=psx+qsy \\&=s(px+qy) \\&=s(x) \end{align} since $px+qy=x$. By letting $s=\frac{1}{x+y}$ we ensure $sx+sy=1$, which means \begin{bmatrix} sx & sy \end{bmatrix} is the equilibrium vector we want.
Hope this helps!