How to understand Markov property?

720 Views Asked by At

I'm learning stochastic process in college. How to understand Markov property?I'm curious about what is the power and validity of Markov property ?

A stochastic process has the Markov property if the conditional probability distribution of future states of the process (conditional on both past and present values) depends only upon the present state, not on the sequence of events that preceded it.

The question is big and perhaps vague. To be specific and clear, I'd like to illustrate some aspects of the question.

Power:

  1. Why we need such property? Why is it essential and ubiquitous? What is the "philosophy" of such property?

Validity:

  1. Is this property an assumption when modelling? If it is, does it hold necessarily? If it does not hold, what's the influence?
2

There are 2 best solutions below

6
On

The underlying model is physical. In physics, everything is local (both in space and time). You wouldn't expect that something that happens on Mars right now would have an immediate influence on you. Similarly with something that happened million years ago. Moreover, it simplifies the modeling, since every vertex of the graph (if you work in the discrete setting) is connected only to few neighbors instead of everything else in the world.

Naturally, in particular empirical model it might turn out that having action at a distance simplifies things. Nevertheless, it is important to understand that any such model is a reduction of a local, physical, Markovian model.

5
On

Why we need such property?

We do not need it, either we assume that it holds or we do not, depending on the phenomenon we seek to model.

Why is it essential and ubiquitous?

Define "essential". Ubiquitous: is it? Markovianity may be used in many situations because it makes computations possible and when the "physics" of a lot of phenomena make it plausible, from what we understand of said phenomena.

What is the "philosophy" of such property?

Define "philosophy". (And the answer is probably: "None".)

Is this property an assumption when modelling?

Yes, one assumes that it holds (or not) hence, when used, this is definitely an assumption.

If it is, does it hold necessarily?

Of course not (this seems to be a matter of pure logic, no?).

If it does not hold, what's the influence?

Define "influence" (and influence on what?).