Let $(E,\mathcal E)$ be a measurable space, $\kappa$ be a Markov kernel on $(E,\mathcal E)$ and $X_I$ denote the projection from $E^{\mathbb N_0}$ onto $E^I$ for $I\subseteq\mathbb N_0$.
We know that for every probability measure $\mu$ on $(E,\mathcal E)$, there is a unique probability $\operatorname P_\mu$ on $(E^{\mathbb N_0},\mathcal E^{\otimes\mathbb N_0})$ with $$\forall n\in\mathbb N_0:\operatorname P_\mu\circ X_{\{0,\:\ldots\:,\:n\}}=\mu\otimes\kappa^{\otimes n}\tag1.$$
Let $\operatorname P_x:=\operatorname P_{\delta_x}$ for $x\in E$ and $\operatorname E_\mu$ denote the expectation with respect to $\operatorname P_x$.
Let $X_n:=X_{\{n\}}$ for $n\in\mathbb N_0$ and $$\theta_k:E^{\mathbb N_0}\to E^{\mathbb N_0}\;,\;\;\;\omega\mapsto(\omega_{k+n})_{n\in\mathbb N_0}$$ for $k\in\mathbb N_0$. We know that $(X_n)_{n\in\mathbb N_0}$ is a strong Markov process on $(E^{\mathbb N_0},\mathcal E^{\otimes\mathbb N_0},\operatorname P_\mu)$ for every probability measure $\mu$ on $(E,\mathcal E)$; i.e. $$\operatorname E_\mu\left[f\circ\theta_\tau\mid\mathcal F_\tau\right]=\operatorname E_{X_\tau}[f]\tag2$$ for every finite stopping time $\tau$ and all bounded and $\mathcal E^{\otimes I}$-measurable $f:E^I\to\mathbb R$. In particular, if $g:E\to\mathbb R$ is bounded and $\mathcal E$-measurable and $n\in\mathbb N_0$, then $$\operatorname E_\mu\left[g(X_{\tau+n})\mid\mathcal F_\tau\right]=(\kappa^{\otimes n}g)(X_\tau)\tag3.$$
Question: How do we obtain the second equality in the first equation of the proof in Proposition 4.4.2 of Markov Chains by Randal Douc, Eric Moulines, Pierre Priouret, Philippe Soulier?
You’re writing the inner conditional expectation as a “regular”expectation with a shift operator. See proposition 3.4.3 of Meyn and Tweedie. This is the (weak) Markov property. Then, you remove one of the extra expectation operators.
Edit: more details:
Your definition (4.4.3) gives you $$ P_A g(x) = \mathbb{E}_x\left[ \mathbb{1}_{\{\tau_A < \infty\} }g(X_{\tau_A}) \right] $$ Thinking of this as a random variable: $$ P_A g(X_1) = \mathbb{E}_{X_1}\left[ \mathbb{1}_{\{\tau_A < \infty\} }g(X_{\tau_A}) \right] . $$
Then using the weak Markov property: $$ P_A g(X_1) = \mathbb{E}_{x}\left[ \mathbb{1}_{\{\tau_A < \infty\} }g(X_{\tau_A}) \circ \theta \mid \mathcal{F}_1 \right] . $$ You're using the weak Markov property because the time shift is deterministic.
Last, when you reintroduce the outer expectation, it's the law of total expectation.