I am trying to solve a continuous-time Markov chain exercise. But I have been stuck for a few days and I don't know how to continue or even If what I have done is correct.
Consider a system having five identical independent components.
Each component works for an exponential time, measured in days, and parameter 4.
When a component breaks down, it is replaced immediately as sufficient technicians are available to address any failure at any time. The duration of a repair is also exponential and on average lasts half a day.
Determines, in the long run, how long the system is out of service because it has all its components damaged.
I consider state n as the state where exactly n components are under repair. So I built the transition matrix Q as below:
$$ Q=\begin{pmatrix} -20 & 20 & 0 & 0 & 0 & 0\\ 2 & -18 & 16 & 0 & 0 & 0\\ 2 & 2 & -16 & 12 & 0 & 0\\ 2 & 2 & 2 & -14 & 8 & 0\\ 2 & 2 & 2 & 2 & -12 & 4\\ 2 & 2 & 2 & 2 & 2 & -10\\ \end{pmatrix} $$
I'm not totally sure that the half under the diagonal is totally correct.
If it is, any hint how to calculate the system availability in the long run?
Let's first assume $j$ components work and $i$ components are being repaired.
The probability that a broken component is fixed before a working component breaks down is $$\mathbb{P}\Big(\min\{T_1,...,T_j\}>\min\{S_1,...,S_i\}\Big)$$ Here $T_1,...,T_j\sim \text{Exponential}(4)$ represent the times each of the $j$ components function properly while $S_1,...,S_i\sim \text{Exponential}(2)$ denote the repair times for each of the i broken components.
To calculate the aforementioned probability, set $T=\min\{T_1,...,T_j\}$ and $S=\min\{S_1,...,S_i\}$. It's easy to show that $T\sim \text{Exponential}(4j)$ and $S\sim \text{Exponential}(2i)$. Since $S,T$ are independent, the joint distribution of $(S,T)$ factors: $$f_{S,T}(x,y)=f_{S}(x)f_{T}(y)=8ij\exp\big\{-(2ix+4jy)\big\}$$ Then $$\mathbb{P}(T>S)=\int_0^{\infty}\int_x^{\infty}f_{S,T}(x,y)\mathrm{d}y\mathrm{d}x=\frac{i}{i+2j}$$
If $X_n$ is the number of components that function properly at time $n$, we see that conditional distribution $X_{n+1}|X_n=j$ for $1\leq j \leq 4$ is supported on $\{j-1,j+1\}$ and has pdf $$\mathbb{P}\left(X_{n+1}=j+1|X_{n}=j\right)=\frac{5-j}{5+j}$$ $$\mathbb{P}\left(X_{n+1}=j-1|X_n=j \right)=\frac{2j}{5+j}$$ Clearly $\mathbb{P}\left(X_{n+1}=1|X_n=0\right)=\mathbb{P}\left(X_{n+1}=4|X_n=5\right)=1$. Since the exponential distribution is memoryless, $\{X_n\}_{n\geq 0}$ is a Markov chain with state transition matrix $$P=\begin{pmatrix}0&1&0&0&0&0\\ \frac{1}{3}&0&\frac{2}{3}&0&0&0\\ 0&\frac{4}{7}&0&\frac{3}{7}&0&0\\ 0&0&\frac{3}{4}&0&\frac{1}{4}&0\\ 0&0&0&\frac{8}{9}&0&\frac{1}{9}\\ 0&0&0&0&1&0\end{pmatrix}$$ The unique stationary distribution of this Markov chain is $$\vec{\pi}=\begin{pmatrix}\frac{8}{81}\\ \frac{8}{27}\\ \frac{28}{81}\\ \frac{16}{81}\\ \frac{1}{18}\\ \frac{1}{162}\end{pmatrix}$$ The expected dwelling time in state $0$ is precisely equal to the expected value of $\min\{S_1,...,S_5\}\sim \text{Exponential}(10)$ which is $\frac{1}{10}$. So from this we see limiting expected holding time in state $0$ is $\frac{1}{10}\pi_0=\frac{4}{405}$.