Competing predators modelled using continuous time Markov Chain.

Question

Imagine we have 3 predators living in the same forest lets call them $P_1$, $P_2$ and $P_3$. They all live by themselves and see eachother as competitors, therefore if one predator meets another one they will fight to death. We assume that all the predators are equally strong such that if two meets in a fight, the probability of one will win against another is $1/2$. We further assume that the waiting time between any two predators meet is exponential distributed with rate parameter $\lambda$. As I see it for each predator there is 4 possible states: "alive", "in a fight with $P_i$", "in a fight with $P_j$" and "dead". It is assumed that all predators start in the "alive" state with probability 1: $$ \phi_i(0) = \begin{pmatrix} 1 \\ 0 \\ 0 \\ 0 \end{pmatrix} \ , \quad i \in \{1,2,3\} $$ I now want to formulate the transition probability matrix $\textbf{P}$. Since there is four states $\textbf{P}$ is a 4 times 4 matrix. The probabilty of $P_k$ meeting $P_i$ at time $t$ must by the exponential distribution assumption be equal to $1-e^{-\lambda t}$, hence we put $\textbf{P}_{21} = 1-e^{-\lambda t}$ and $\textbf{P}_{31} = 1-e^{-\lambda t}$. Further since we assume that a predator only can die from a fight we put $\textbf{P}_{41} = 0$. Now since each columns in the transition probability matrix has to add to one the first column in $\textbf{P}$ must be: $$ \textbf{P}^{(1)} = \begin{pmatrix} 2e^{-\lambda t} - 1 \\ 1-e^{-\lambda t} \\ 1-e^{-\lambda t} \\ 0 \end{pmatrix} $$ The second and third column in $\textbf{P}$ corresponds to a predator in a fight in another predator. I imagine that the fight is instantaneous in time therefore these columns have the structure: $$ \textbf{P}^{(2)} = \textbf{P}^{(3)} = \begin{pmatrix} \frac{1}{2} \\ 0 \\ 0 \\ \frac{1}{2} \end{pmatrix} $$ The last column corresponds to the case there the predator is dead, and if it is dead it way stay in that state forever thus: $$ \textbf{P}^{(4)} = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 1 \end{pmatrix} $$ From this we conclude that the total probability transition matrix must have the from: $$ \textbf{P}(t) = \begin{pmatrix} 2e^{-\lambda t} - 1 & \frac{1}{2} & \frac{1}{2} & 0 \\ 1-e^{-\lambda t} & 0 & 0 & 0\\ 1-e^{-\lambda t} & 0 & 0 & 0\\ 0 & \frac{1}{2} & \frac{1}{2} & 1 \end{pmatrix} \ . $$ We now want to find the probability for being in each of the states for each predator. To find this be multiply the initial state with the transition matrix then we get: $$ \phi_i(t) = \textbf{P}(t)\phi_i(0) = \begin{pmatrix} 2e^{-\lambda t} - 1 & \frac{1}{2} & \frac{1}{2} & 0 \\ 1-e^{-\lambda t} & 0 & 0 & 0\\ 1-e^{-\lambda t} & 0 & 0 & 0\\ 0 & \frac{1}{2} & \frac{1}{2} & 1 \end{pmatrix} \begin{pmatrix} 1 \\ 0 \\ 0 \\ 0 \end{pmatrix} = \begin{pmatrix} 2e^{-\lambda t} - 1 \\ 1-e^{-\lambda t} \\ 1-e^{-\lambda t} \\ 0 \end{pmatrix} \ . $$ This final results seems wrong since for large $t$ we get: $$ \lim_{t \to \infty} \phi_i(t) = \begin{pmatrix} - 1 \\ 1 \\ 1 \\ 0 \end{pmatrix} $$ which is not a probability vector. Is my approach wrong?

Answer 1

There are a few different things here that seem confusing!

(1) Although it may be plausible to model the whole system as a Markov chain, that doesn't mean that the "state" of each individual is Markov. (For example, thinking about the state of a single predator P1 alone: suppose you know that P1 is "alive" at some given time $t$; if you also know that they have already fought with P2 and with P3, then you know they will now live for ever. So the path we took to the current state could influence the future evolution, meaning that the process is not Markov.)

(2) If you "imagine that the fight is instantaneous in time" then you don't need states in which fights are occurring, since the chain never spends any time in those states.

(3) Something weird is going on with the way you're setting up your "transition matrix". Perhaps you're mixing together how things work in continuous time and in discrete time? It sounds like you're intending your "$\mathbf{P}$" to represent the transition matrix between times $0$ and $t$. But that matrix only seems to be taking into account the possibility of one jump during that time interval, whereas of course there could be several. To set things up in continuous time (as your title suggests you want to), you normally start with the jump rates, which give you your "$Q$-matrix" (or "generator") $Q$. Then the time-$t$ transition matrix is given by $P_t=e^{tQ}$.

So we could set up a chain whose states record which predators are alive at a given time. Say "123", "12", "13", "23", "1", "2", "3". According to your description, for every pair of predators alive, each kills the other at rates $\lambda/2$. So your chain has transitions:

• from "123" to "12", to "13", and to "23";
• from "12" to "1" and to "2";
• from "13" to "1" and "3";
• from "23" to "2" and "3".

Out of "123", each of the 3 possible transitions happens at rate $\lambda$, and out of "12", "13", "23", each of the 2 possible transitions happens at rate $\lambda/2$.

From this you could form your $Q$-matrix and take things from there. But in fact the chain is quite simple. The path will have precisely two jumps. It starts will all three predators alive. After an $\mathrm{Exp}(3\lambda)$-distributed time, it jumps to a state with two predators alive. Then after a further (independent) $\mathrm{Exp}(\lambda)$-distributed time, it jumps to a state with just one predator alive, and stays there for ever. So if you want to do calculations, it may be easier to do so using independent random variables $X\sim\mathrm{Exp}(3\lambda)$, $Y\sim\mathrm{Exp}(\lambda)$ rather than via the set-up with the $Q$-matrix.