Probability of state as a function of time with random waits between transitions

24 Views Asked by At

I'd like to compute the probability that an asset is at a node in a network as a function of time if there is a wait time distribution at each node and the connections are probabilistic.

For example, consider a network of three nodes: $A, B, C$. Node $A$ connects to $B$ with a 30% probability and to $C$ with a 70% probability. $B$ and $C$ are not connected.

Each node has a wait time modeled as a separate distribution. After the wait at its current node, the asset proceeds to the next connected node, choosing which based on the probability. After waiting at $B$ or $C$, the asset goes home $H$, so perhaps that is another node, but home only receives assets and that's where we stop tracking.

If an asset is delivered to node $A$ at time t=0, can I write the probability of being in each location as a function of time? $P_A(t), P_B(t), P_C(t), P_H(t)$?

In my attempts to solve this, I've determined that if I convolve the wait time distribution at node $A$ with that at node $B$, I can get the wait time of $A+B$. This has led me down the rabbit hole of trying to find distributions that capture wait time that behave under convolution. Gamma is nice if the scales/rates match. $$Gamma(A+B,scale) \sim Gamma(A,scale) \circ Gamma(B,scale)$$

If I turn this joint wait time into a cdf, I can get the probability of leaving (or staying) at $B$ as a function of time given it already waited at $A$. Do I multiply this by the probability of leaving $A$ to get just the probability of being at $B$ (also multiplying by the probability of moving from $A$ to $B$)?

This is almost like a Markov process, but I don't think what I describe is Markovian. My node exit probabilities can change depending on how the node was entered (not in this simple example) and my wait times only start when you enter the node. That said, should I be looking for ways to cast my problem as Markovian? Would that help solve it?