Markov Property and its meaning/allusion

473 Views Asked by At

Starting Markov processes so I'm kind of new to this; I would like someone to explain the property please.
I was given that
Definition: A Markov process has
$$\mathbb{P}\left(X(t_n)\leq x_n |X(t_1)=x_1,X(t_2)=x_2,\ldots,X(t_{n-1})=x_{n-1}\right)\\ = \mathbb{P}(X(t_n)\leq x_n | X(t_{n-1})=x_{n-1}). $$
In a markov chain, given the present state, the past states do not have influence on the future.

I'm not sure what this is alluding to. It seems that Markov processes will then use random variables that are exponentially distributed (since they are known to be memoryless)?
Is the property saying that, if we wanted to find the probability of an event being less than some observed value on the n'th observation, this is only conditioning on the previous observation, and not say, the previous previous observation?

1

There are 1 best solutions below

0
On BEST ANSWER

To answer your last sentence: yes, almost exactly. Perhaps it'd be better to say that the probability of some state can be determined by knowing ONLY the previous state. (It might also be determinable from knowing the second-to-last one, but the key thing is that you don't NEED that information.)

As an example: a MP with only one state, $x_1$, has the property that the current state can be predicted by know the state two periods before, or knowing it one period before.

A second example: a two-state MP, in which the transition matrix is the identity, i.e., if $X(t_{n-1}) = A$, then $X(t_n) = A$, has that same property.

But in general, the point is that the probability of a particular state can be determined by knowing ONLY the previous state (and the transition probabilities).

It's probably a little easier to learn about Markov-ness by looking at finite-state or discrete examples before you move to continuous one. When you move to continuous state stuff, you have to worry about probability-zero events, etc., and the possibility that a random variable might be netter expresses by a CDF than a PDF, etc., and it becomes easier to express this "just remember what happened yesterday" kind of idea with "less than or equal" rather than "equal", but it also can obscure some of the simplicity.