There is a n by n probability matrix $Q = Q_{ij}$ and $Q_{ij} \ge 0 \quad(i, j = 1,2,...,n)$ with $\sum_{j=1}^nQ_{ij} = 1 \quad (j = 1,2,...,n)$.
I am curious about how to prove that $Q^2 = QQ$ is still a probability matrix, and I have tried to use the deduction method below,
n = 2 $$ \begin{align*} Q^2 & = QQ = \begin{pmatrix} q_{11} & q_{12} \\\ q_{21} & q_{22} \end{pmatrix} \begin{pmatrix} q_{11} & q_{12} \\\ q_{21} & q_{22} \end{pmatrix} \\\ & = \begin{pmatrix} q_{11}^2 + q_{12}q_{21} & q_{11}q_{12} + q_{12}q_{22} \\\ q_{21}q_{11} + q_{22}q_{21} & q_{21}q_{12} + q_{22}^2 \end{pmatrix} \\\ \end{align*} $$
From above, I can clearly see that the sum of rows remains at $1$ if I add each row:
$$ \begin{align*} q_{11}^2 + q_{12}q_{21} + q_{11}q_{12} + q_{12}q_{22} & = q_{11}(q_{11} + q_{12}) + q_{12}(q_{21} + q_{22}) \\\ & = q_{11}\cdot1 + q_{12} \cdot 1 = q_{11} + q_{12} = 1 \end{align*} $$
I think this can work out, but I am not sure if I should use this way since I think there is a better way than this, like from the property of stochastic matrix that always has eigenvalue $1$.
Can anyone give me some advice about thinking about this? Thank you in advance.
Define $v=(1,\dots,1)^T$. $Q$ is a probability matrix if and only if its entries are all nonnegative and $Qv = v$. This second condition is a restatement of the fact that all rows must sum to one.
Suppose $Q_1$ and $Q_2$ are probability matrices. Multiplying and adding nonnegative numbers gives you nonnegative numbers. So $Q_1Q_2$ has nonnegative entries. Furthermore: $$(Q_1Q_2)v = Q_1(Q_2v)=Q_1v=v$$
So, $Q_1Q_2$ is also a probability matrix.