A machine consists of $K$ components in series, i.e., all the components must be in working condition for the machine to be functional. When the machine is functional at the beginning of the $n$th day, each component has a probability $p$ of failing at the beginning of the next day, independent of other components. (More than one component can fail at the same time).
When the machine fails, a single repair person repairs the failed components one by one. It takes exactly one day to repair one failed component. When all the failed components are repaired the machine is functional again, and behaves as before. When the machine is down, the working components do not fail.
Let $X_n$ be the number of failed components at the beginning of the $n$th day, after all the failure and repair events at that time are accounted for. Show that $\{X_n, n>0\}$ is a Discrete time Markov chain. Display the transition matrix or the transition diagram.
I don't know how to start this problem. I didn't understand the problem.
Here we have $$ X_{n+1} = \begin{cases} X_n + 1,& 1\leqslant X_n< K\\ V_{n+1}, & X_n = K, \end{cases} $$ where the sequence $\{V_n:n=1,2,\ldots\}$ is independent with $\operatorname{Bin}(K,1-p)$ distribution. Since $X_{n+1} = f(X_n, V_{n+1})$ where $V_{n+1}$ is independent of the $X_n$, we readily see that $\{X_n:n=0,1,\ldots\}$ satisfies the Markov property. The transition probabilities are given by $$ p_{ij} = \begin{cases} 1,& j=i+1,\ 0<i<K\\ \binom K {j-1} (1-p)^{j-1} p^{K-j+1},& i=K,\ 0<j<i\\ (1-p)^K + K(1-p)^{K-1}p,& i=j=K. \end{cases} $$ The subtlety here is that there are two ways the system can transition from $i$ to $j$ - either there are no failures, or there is exactly one failure (in which case it is repaired in that same period).