Is it correct to say a state-space model has the Markov property?

Question

in control theory, there exists a mathematical model of a control loop, called the state-space model. It allows for the computation of a state vector $x$ at discrete time $k$. For the computation, it requires knowledge of the previous state vector at $k-1$ only.

So, I wonder: Is it correct to say the state-space model has the Markov property? Markov property is also called memoryless property, but it is defined in the context of probability distributions. So I am not sure if my statement is valid.

Answer 1

A general state space model can be very complicated. Dynamical systems with hysteresis will definitely not have the Markov property.

If you can write down your system as

$$x_{k+1} = f(x_k,u_k)$$

then it will have the Markov property because only the current state and the current action/actuation are important for the next state.

Answer 2

Markov property is related to the probabilistic model of the state-space equations. $$ \begin{array}{l} {{\bf{x}}_k} = {{\bf{f}}_{k - 1}}({{\bf{x}}_{k - 1}},{{\bf{u}}_k}) + {{\bf{v}}_k} \Leftrightarrow p({{\bf{x}}_k}|{{\bf{x}}_{k - 1}},{{\bf{u}}_k})\\ {{\bf{y}}_k} = {{\bf{h}}_k}({{\bf{x}}_k},{{\bf{u}}_k}) + {{\bf{w}}_k} \Leftrightarrow p({{\bf{y}}_k}|{{\bf{x}}_k},{{\bf{u}}_k}) \end{array}$$ where $\bf{v}_k$ , $\bf{w}_k$ are process and measurement noise, respectively. This state–space representation which incorporate random inputs or noise sources along with random initial conditions, is called Gauss–Markov model.

Under the first-order Markov assumption, conditional probability distribution of future states of the process (conditional on both past and present states) depends only upon the present state (2).

Since any pth-order Markov process can be transformed to a first-order process (1), (3), then we say that any state-space model has first-order Markov property.