I am having some difficulty understanding some early results of Markov Chain theory on a general state space.
We have a function (Kernel) $K:E \times E \rightarrow \mathbb{R}$, and a distribution $\mu(x)$ such that
$P(X_{t} \in A\mid X_{t-1} = x) = \displaystyle\int\limits_{A}K(x,y)\text{d}y$, $ \ \ \text{and} \ \ $ $P(X\in A) = \displaystyle\int\limits_{A}\mu(x)\text{d}x$,
where $E$ is the set of all possible values of our chain.
Here is where I get stuck.
It then follows that
$P(X_{1}\in A_{1},..., X_{t}\in A_{t}) = \displaystyle\int\limits_{A_{1}\times\cdots\times A_{t}} \mu(x_{1})\prod\limits_{k=2}^{t}K(x_{k-1},x_{k})\text{d}x_{1}...\text{d}x_{t}$.
Which I am not quite able to follow through.
My attempt:
$P(X_{1}\in A_{1},..., X_{t}\in A_{t}) = P(X_{1}\in A_{1})\displaystyle\prod\limits_{k=2}^{t}P(X_{k}|X_{1}\in A_{1},..., X_{k-1}\in A_{k-1})$
$\hspace{4.7cm}= \displaystyle\int\limits_{A_{1}}\mu(x_{1})\text{d}x_{1}\prod\limits_{k=2}^{t}P(X_{k}|X_{k-1}\in A_{k-1})$,
which is almost what I am after.. except for the fact that I have a condition on $X_{k-1}\in A_{k-1}$, whereas the definition of $K$ is conditioned on $X_{t-1} = x_{t-1}$. And I am a little unsure what to do with the order of integration.
I hope this makes sense.
Thanks for your insights and thoughts.
The notation is not very clear, so I am making it a bit more standard in my answer.
Let's look at two periods \begin{eqnarray*} \Pr \left[ X_1 \in A_1 \right] & = & \int_{A_1} \mathrm{d} \mu \left( x_1 \right) \end{eqnarray*} \begin{eqnarray*} \Pr \left[ X_1 \in A_1, X_2 \in A_2 \right] & = & \int_{A_1} \Pr \left[ X_2 \in A_2 |X_1 = x_1 \right] \mathrm{d} \mu \left( x_1 \right) \end{eqnarray*} But \begin{eqnarray*} \Pr \left[ X_2 \in A_2 |X_1 = x_1 \right] & = & k \left( x_1, A_2 \right)\\ & = & \int_{A_2} k \left( x_1, \mathrm{d} x_2 \right) \end{eqnarray*} Thus \begin{eqnarray*} \Pr \left[ X_1 \in A_1, X_2 \in A_2 \right] & = & \int_{A_1} \int_{A_2} k \left( x_1, \mathrm{d} x_2 \right) \mathrm{d} \mu \left( x_1 \right) \end{eqnarray*} By induction, using the Markov property (for every $t$, $\Pr \left[ X_t \in A_t |X_1 \in A_1, \ldots, X_{t - 1} \in A_{t - 1} \right]= \Pr \left[ X_t \in A_t |X_{t - 1} \in A_{t - 1} \right]$), you could deduce that \begin{eqnarray*} \Pr \left[ X_1 \in A_1, \ldots, X_{t - 1} \in A_{t - 1}, X_t \in A_t \right] & = & \int_{A_1 \times \cdots \times A_{t - 1}} k \left( x_{t - 1}, A_t \right) k \left( x_{t - 2}, \mathrm{d} x_{t - 1} \right) \cdots k \left( x_1, \mathrm{d} x_2 \right) \mathrm{d} \mu \left( x_1 \right)\\ & = & \int_{A_1 \times \cdots \times A_{t - 1} \times A_t} k \left( x_{t - 1}, \mathrm{d} x_t \right) k \left( x_{t - 2}, \mathrm{d} x_{t - 1} \right) \cdots k \left( x_1, \mathrm{d} x_2 \right) \mathrm{d} \mu \left( x_1 \right)\\ & & \end{eqnarray*}
Since you are having trouble understanding the reasoning, here is an over-detailed explanation without the induction argument
\begin{eqnarray*} \Pr \left[ X_1 \in A_1, X_2 \in A_2, X_3 \in A_3 \right] & = & \int_{A_1} \Pr \left[ X_3 \in A_3, X_2 \in A_2 |X_1 = x_1 \right] {d} \mu \left( x_1 \right)\\ & = & \int_{A_1} \underbrace{\Pr \left[ X_3 \in A_3 |X_2 \in A_2, X_1 = x_1 \right] \Pr \left[ X_2 \in A_2 |X_1 = x_1 \right]}_{= \Pr \left[ X_3 \in A_3 |X_2 \in A_2 \right] \Pr \left[ X_2 \in A_2 |X_1 = x_1 \right]} {d} \mu \left( x_1 \right)\\ & = & \int_{A_1} \left( \int_{A_2} \Pr \left[ X_3 \in A_3 |X_2 = x_2 \right] \Pr \left[ X_2 \in \mathrm{d} x_2 |X_1 = x_1 \right] \right) {d} \mu \left( x_1 \right)\\ & = & \int_{A_1} \left( \int_{A_2} \Pr \left[ X_3 \in A_3 |X_2 = x_2 \right] k \left( x_1, \mathrm{d} x_2 \right) \right) \mathrm{d} \mu \left( x_1 \right)\\ & = & \int_{A_1 \times A_2} \underbrace{\Pr \left[ X_3 \in A_3 |X_2 = x_2 \right]}_{= k \left( x_2, A_3 \right)} k \left( x_1, \mathrm{d} x_2 \right) \mathrm{d} \mu \left( x_1 \right)\\ & = & \int_{A_1 \times A_2} \left( \int_{A_3} k \left( x_2, \mathrm{d} x_3 \right) \right) k \left( x_1, \mathrm{d} x_2 \right) \mathrm{d} \mu \left( x_1 \right)\\ & = & \int_{A_1 \times A_2 \times A_3} k \left( x_2, \mathrm{d} x_3 \right) k \left( x_1, \mathrm{d} x_2 \right) \mathrm{d} \mu \left( x_1 \right) \end{eqnarray*}
The second line follows by Markov.
To progress further, it is a good investment to learn measure theory (e.g. Rosenthal's "A First Look at Rigorous Probability Theory".