I encountered a question about Markov Chains which looks interesting.
Given a homogeneous, irreducible, non cyclic Markov Chain with $K$ possible states and a transition matrix $Q$.
We define $T_i$ to be a random variable which is the minimal time ($t\geq1$) in which the Chain reaches a state $i$.
We also define a $K\times K$ matrix $M$ as follows: $M_{ij}=E[T_j|X(0)=i]$.
We define $C$ as a matrix of ones, and $D$ a diagonal matrix: $D_{ii}=M_{ii}$.
Prove that $M=C+Q(M-D)$.
Can someone give me a hint on how to start solving this? I don't even know where to begin.