Let $(\Omega, \mathcal F,\mathbb P)$ be the probability space. Let $N \in \mathbb{N^*}$ and $(X_n)_{n \in \mathbb N}$ be a sequence of random variables with values in $\{1, ..., N\}$. Let $\mathcal F_n = \sigma(X_0, ..., X_n)$ for all $n \in \mathbb N$ define the associated filtration. We assume the following :
- (A) $\forall {n \in \mathbb N},\ \forall k_0, ..., k_{n+1} \in \{1, ..., N\}$,
$$\mathbb P(X_{n+1}=k_{n+1}|X_n=k_n) = \mathbb P(X_{n+1}=k_{n+1}|\bigcap\limits_{j=0,...,n}(X_j=k_j) )$$
- (B) $\forall j,k \in \{1, ..., N\},\ \forall {n \in \mathbb N}$,
$$ p_{j,k}:=\mathbb P(X_1=k|X_0=j) = \mathbb P(X_{n+1}=k|X_n=j)$$
- (C) $\forall j,k \in \{1, ..., N\}$,
$$ \mathbb P(\exists n \in \mathbb N, X_n=k|X_0=j)=1$$
Let $T=(p_{j,k})_{1≤j,k≤N}, (e_j)^N_{j=1}$ be the canonical basis of $\mathbb R^N$. Define $p^n_{j,k}:=\mathbb P(X_n=k|X_0=j)$ for all $n \in \mathbb N$.
Prove that $\forall j,k \in \{1, ..., N\},\ \forall n \in \mathbb N$,
$$\ p^n_{j,k}=\langle T^n e_k,e_j \rangle$$
Note: Please do not quote results from the theory of Markov chains directly.
My proof is :
"$(e_j)^N_{j=1}$ is the canonical basis of $\mathbb R^N$" means that $(e_j)^N_{j=1}$ is the standard basis (By wiki, Canonical basis refers to the standard basis in a coordinate space and $\mathbb R^N$ is real coordinate space), "$T=(p_{j,k})_{1≤j,k≤N}$" means that "$T^n=p^n_{j,k}$",so we can easily get that $\langle T^n e_k,e_j \rangle =T^n=p^n_{j,k}$ by method of inner product calculation with Euclidean $n$-space.
Actually, I believe that this proof is too simple (maybe it's not right at all) and there must be something missing in this proof (I even do not use any given information about conditional probability as I do not know how to use this information). Does anyone know how to improve this proof? Thank you so much!
Based on your (A),(B) and (C), is Markov chain, and $T$ is the transition matrix, note distribution $\mu$ transition is $\mu T$, hence $p_{j,k}^n= e_j'T^n e_k= \langle T^ne_k,e_j\rangle$.