If $X_n$ is the remaining time after $n$ until the next replacement, show that $(X_n)$ is a Markov chain and compute its transition probabilities

Question

Problem. Consider a certain workpiece of equipment that is currently in use. When workpiece fails is immediately replaced by another identical. When this other failure, is also replaced, and so on. Let the probability that a workpiece lasts exactly $k$ time units ($k = 1,2,3, \ldots$) equal to $p_k$.

Define $X_n$ the time as the remaining life of the workpiece at time $n$. I would like to prove that $\{X_n\}$ is a Markov chain, i.e. $$ P(X_{n+1}=j|X_1=i_1,\ldots,X_n=i_n)=P(X_{n+1}=j|X_n=i_n) $$ and determine its transition probabilities $$ P(X_{n+1}=j|X_n=i_n). $$

My attempt. Just for attaching the notation, let $T = \{0,1,2,3,\ldots\}$ be the set parameters of the stochastic process in question, i.e. the time instant in which it checks whether the workpiece needs to be replaced or not . Is still $\mathcal{S} = \{\ldots,-3, -2,-1,0,1,2,3,\ldots\}$ the set of states that indicates the lifetime (remaining) part observed by the random variable $X_n$.

I found it wise to consider the instants of the time $0, -1, -2, -3, \ldots $ in the state space. The reason is for a possible general deemed necessary in solving the problem. Thus, for example, $X_n = -3$ mean that the timelife of workpiece ended at time $n-3$. And $X_n = 0$ means that the workpiece lifetime ended at time $n$.

Define an auxiliary random variable $V_n =\mbox{ lifetime of workpiece replaced at time } n$. From the hypothesis of the problem we have to $$ P(V_n=k)=p_k. $$ I divide the problem and two cases.

• case(1): the workpiece is replaced at time $n$. And so we have $X_n=V_n$ and $$ P(X_n=i_n)=P(V_n=i_n). $$
• case(2): the workpiece is replaced before the time $n$. In this second case, $X_n \neq V_n$

I have calculated the probabilities $P(X_{n+1}=j|X_n=i_n)$ only in case (1) in which the workpiece is replaced at time $n$. To this I divide the case (1) in three subcases. $$ \begin{array}{lll} (1.a) & P(X_{n+1}=j_{_<}|X_1=i_1,\ldots,X_n=i_n)=P(X_{n+1}=j_{_<}|X_n=i), & \mbox {for } j_{_<}+1<i, \\ (1.b) & P(X_{n+1}=j_{_=}|X_1=i_1,\ldots,X_n=i_n)=P(X_{n+1}=j_{_=}|X_n=i), & \mbox {for } j_{_=}+1=i, \\ (1.c) & P(X_{n+1}=j_{_>}|X_1=i_1,\ldots,X_n=i_n)=P(X_{n+1}=j_{_>}|X_n=i), & \mbox {for } j_{_>}+1>i. \end{array} $$ For the calculations I used the diagram below. In the top band we have the states $j_{_<}, j_{_=}, j_{_>}$ that the random variable $X_{n+1}$ takes and the state $i_n$ that the random variable $X_n$ takes. In the lower band we have a timeline that represents the parameter set T.

In this diagram is easy to see that

$$ \begin{array}{ll} P(X_{n+1}=j_{_<}|X_n=i)=0, & \mbox {for } j_{_<}+1<i \mbox{ and } X_n=V_n, \\ P(X_{n+1}=j_{_=}|X_n=i)=1, & \mbox {for } j_{_=}+1=i \mbox{ and } X_n=V_n, \\ P(X_{n+1}=j_{_>}|X_n=i)=p_{j_{_>}+1-i_n}, & \mbox {for } j_{_>}+1>i \mbox{ and } X_n=V_n. \end{array} $$ However I have no idea how to check the equalities (1.a), (1.b) and (1.c).

For case (2) I'm looking for an algebraic relationship between $V_n$ and $X_n$ to allow me to apply the principle of substitution. The difficulty is that the time instant when the workpiece is replaced it is not deterministic.

Question. How to prove that $\{X_n\}$ is a Markov chain? How to calculate the probabilities below? $$ \begin{array}{ll} P(X_{n+1}=j_{_<}|X_n=i)=?, & \mbox {for } j_{_<}+1<i \mbox{ and } X_n\neq V_n, \\ P(X_{n+1}=j_{_=}|X_n=i)=?, & \mbox {for } j_{_=}+1=i \mbox{ and } X_n\neq V_n, \\ P(X_{n+1}=j_{_>}|X_n=i)=?, & \mbox {for } j_{_>}+1>i \mbox{ and } X_n\neq V_n. \end{array} $$ Any idea?

Thank you.

Answer 1

Let $\{T_k \}_{k=0}^{\infty}$ be the sequence of (random) times when the machine is replaced, and let $\{V_j\}$ be defined as above so that $V_{T_k} = $ the lifetime of the $k^{th}$ replacement, $$ T_k = \sum_{j=0}^{k-1} V_j. $$ Then the Markov property should be clear since $P(X_{n+1} = j ~|~ X_n = i_n, \dots, X_1 = i_1, X_0 = V_0)$ and $P(X_{n+1} = j ~|~ X_n = i_n ) $ are both equal to

$$ \left\{ \begin{array}{cc} \delta(j,i_n - 1) & \textrm{ if } ~ n+1 \notin \{T_k\} \\ p_j & \textrm{ if } ~ n+1 \in \{T_k\} \end{array} \right. $$ where $\delta(\cdot,\cdot)$ is a Kronecker delta function.

Answer 2

If I may I also took the liberty of answering my question. For me is a way to give a feedbeck. Following the notation in the answer given by Titus, we have to \begin{align} P(X_{n+1}|X_n=i_n,\ldots, X_{1}=i_{1},X_{0}=i_0) =& P(X_{n+1}|X_n=i_n,\ldots, X_{1}=i_{1},X_{0}=i_0, V_0=X_0) \\ \end{align} Let $\tau_n=\max\{ T_k : T_k\leq n+1\}$. If $\tau_n=n+1$ then $[X_0=i_0,\cdots,X_n=i_n]$ and $[X_{n+1}=j]$ are independents and $$ P(X_{n+1}=j|X_{0}=i_0,\dots,X_{n}=i_{n})= P(X_{n+1}=j|X_{n}=i_n)=P(X_{n+1}=j)=P(V_{n+1}=j)=p_j. $$ If $\tau_n\leq<n$ then the event $ [X_n=i_n,X_{n-1}=i_{n-1},\ldots,X_{1}=i_{1},X_{0}=i_{0}] $ is the union of the independents events below $$ \begin{array}{l} [X_{n}=i_{n},\ldots,X_{\tau_{n}}=i_{\tau_{n}}] =[X_{n}=i_{n},\ldots,X_{\tau_{n}}=i_{\tau_{n}},V_{\tau_n}] \\ [X_{\tau_n-1}=i_{\tau_n -1},\ldots,X_{\tau_{n-1}}=i_{\tau_{n-1}}, V_{\tau_{n-1}}=X_{\tau_{n-1}}] \\ \hspace{2cm}\vdots \\ [X_{T_k-1}=i_{T_k-1},\ldots,X_{T_{k}}=i_{T_{k}}] [X_{T_k-1}=i_{T_k-1},\ldots,X_{T_{k}}=i_{T_{k}},V_{T_{k}}=X_{T_{k}}] \\ \hspace{2cm}\vdots \\ [X_{T_0-1}=i_{T_0-1},\ldots,X_{T_{0}}=i_{T_{0}}] = [X_{T_0-1}=i_{T_0-1},\ldots,X_{T_{0}}=i_{T_{0}},V_{T_{0}}=X_{T_{0}}] \end{array} $$ Here $ T_0=0$. Therefore, $$ P(X_{n+1}|X_n=i_n,\ldots, X_{1}=i_{1},X_{0}=i_0) = P(X_{n+1}|X_n=i_n,\ldots,X_{\tau_n+1}=i_{\tau_n+1}, V_{\tau_n}=X_{\tau_n}) $$ Since the random variables $X_{\tau}, X_{\tau+1},\ldots, X_n$ has the following relationship $ X_{p}+1=X_{p-1}, $ for $ p=\tau_n,\tau_n+1,\ldots, n$, we have $[X_n=i_n,\ldots,X_{\tau_n+1}=i_{\tau_n+1}, V_{\tau_n}=X_{\tau_n}]=[X_n=i_n]$. Then $$ P(X_{n+1}=j|X_n=i_n,X_{n-1}=i_{n-1},\ldots,X_{\tau+1}=i_{\tau+1},X_{\tau}=i_{\tau},V_{\tau}=X_{\tau})=P(X_{n+1}=j|X_n=i_n) $$ and $$ P(X_{n+1}=j|X_n=i_n)=\begin{cases} 1, & \mbox{ if }, i=j \\ 0, & \mbox{ if } i \neq j \end{cases} $$