Suppose we have a stochastic process in discrete time that we can model by means of a single state variable $X_n$, with $n$ the discretized time. If the value of the immediate future state $X_{n+1}$ stochastically depends only on the value of our current state $X_n$, then we have a Markov process.
But what if we want to give the process a longer "memory"? More concretely, I wish to extend the Markov process by imposing conditional constraints on the transition probabilities.
By conditional constraints I mean constraints of the following form:
Let the process's current state $X_n$ be $B$. Then the probability of the next state $X_{n+1}$ being $C$ equals $$ P(X_{n+1} = C | X_n = B) = \cases{p_1 \text{ if the previous state $X_{n-1}$ was $A_1$}\\ p_2 \text{ if the previous state $X_{n-1}$ was $A_2$}. \\ \vdots} $$
I will suppose that your state space is discrete and I name it $E$, but everything can be generalized.
Now, assume you want some "second order" Markov chain $(X_n)_{n\geq 0}$ with two steps of memory. In addition, you want that the probability to obtain $X_n$ at state $a$ knowing you had at $X_{n-1}$ state $b$ and $X_{n-2}$ at state $c$ is given by $p(c,b,a)$ for some function $p:E^3\to[0,1]$.
To do so, you can use the following procedure: consider a Markov chain $Z_n$ with state space $E^2$ and transition given by $$\mathbb{P}(Z_n=(x_1,y_1) \mid Z_{n-1}=(x_2,y_2))=p(y_2,x_2,x_1)1_{y_1=x_2}$$ and set $X_n$ to be $\pi(Z_n)$ where $\pi$ is the projection along the first coordinate. In other words, $Z_n=(X_n,X_{n-1})$ and you will have $$\mathbb{P}(X_n=a \mid X_{n-1}=b, X_{n-2}=c)=\mathbb{P}(Z_n=(a,b) \mid Z_{n-1}=(b,c))=p(c,b,a)$$ which is what you want.