Let $X$ and $Y$ be two finite state spaces.
We consider an irreducible Markov chain $(Y_n)$ on $Y$. Let $(q_{j,k})_{j,k}$ be the associated transition matrix.
We consider furthermore a Markov chain $(X_n,Y_n)$ on $X\times Y$, the transition matrix of which is given by $(p_{(i,j),l} q_{j,k})$, i.e. the transition on $Y$ is given by $(q_{j,k})_{j,k}$ whereas the transition on $X$ depends both on $X_n$ and on $Y_n$.
Finally, we suppose that every Markov chain induced by $(X_n,Y_n)$ on $X$ is irreducible and aperiodic.
By induced we mean a Markov chain on $X$ the transition of which is given by $\tilde{p}_{i,l} = \sum_{j\in Y} m^i_j p_{(i,j),l}$ with $m^i_j\geq 0$ and $\sum_{j\in Y} m^i_j = 1$ for all $i\in X$.
We want to prove that the Markov chain $(X_n,Y_n)$ is irreducible. I cannot find a proof but I cannot manage to get a counterexample either.