I'm reading the paper On forward improvement iteration for stopping problems by Albrecht Irle:
We consider a discrete homogeneous Markov process $(Z_{n})$ with respect to the underlying filtration. The measurable state space $(S, \mathcal{S})$ is finite. Let $g: S \rightarrow \mathbb{R}$ be measurable and $\alpha \in (0,1]$. We look at the optimal stopping problem for $$X_{n}=\alpha^ng\left(Z_{n}\right)$$
Let
- $P_{z}, E_{z}$ denote $P(\cdot \mid Z_{0}=z), E(\cdot \mid Z_{0}=z)$ respectively.
- $E_{z} X_{\tau}$ exist for all stopping rules $\tau$ and all $z \in S$.
- $p_{z y}=P(Z_{1}=y \mid Z_{0}=z)$ for all $y, z \in S$.
- $\tau_{n}(B)=\inf \left\{j \geq n \mid Z_{j} \in B\right\}$ for a measurable $B \in \mathcal{S}$.
- $h_{i}(B)(z)=E_{z} \alpha^{\tau_{i}(B)} g\left(Z_{\tau_{i}(B)}\right)$ for $z \in S$ and $i \in \{0,1\}$.
Then the author present a proposition and its proof:
I'm trying to understand how to get $$\forall z \in S \setminus B:h(z)=\color{blue}{\alpha} \sum_{y} p_{z y} h(y)$$ We have\begin{aligned} h_{0}(B)(z) &= E_z \left [ \alpha^{\tau_{0}(B)} g\left(Z_{\tau_{0}(B)}\right) \right ]\\ &= \sum _{k=0}^\infty \alpha^{k} g\left(Z_{k}\right) P_z\left [ \tau_{0}(B) = k \right ]\\ &= \sum _{k=0}^\infty \alpha^{k} g\left(Z_{k}\right) \sum_y P_z\left [ \tau_{0}(B) = k,X_1=y \right ] \\ &= \sum _{k=0}^\infty \alpha^{k} g\left(Z_{k}\right) \sum_y P_z\left [ \tau_{0}(B) = k \mid X_1=y \right ] P_z [X_1=y]\\ &= \sum _{k=0}^\infty \alpha^{k} g\left(Z_{k}\right) \sum_y P_y\left [ \tau_{0}(B) = k \right ] p_{zy}\\ &= \sum_y \left [ \sum _{k=0}^\infty \alpha^{k} g\left(Z_{k}\right) P_y\left [ \tau_{0}(B) = k \right ] \right ] p_{zy}\\ &= \sum_y E_y \left [ \alpha^{\tau_{0}(B)} g\left(Z_{\tau_{0}(B)}\right) \right ] p_{zy}\\ &= \sum_y h_{0}(B)(y) p_{zy} \end{aligned}
In my attempt, I could not see how the constant $\color{blue}{\alpha}$ appears. Could you please elaborate on this point?
$\def\peq{\mathrel{\phantom{=}}{}}\def\tb{{τ_0(B)}}$Since $z \in S \setminus B$, then\begin{align*} &\peq E_z\left( α^\tb g(Z_\tb) \right) = \sum_{k = \color{red}{1}}^∞ E_z\left( α^k g(Z_k) I_{\{\tb = k\}} \right)\\ &= \sum_{k = 1}^∞ \sum_{\color{red}{x \in B}} E_z(α^k g(x) I_{\{Z_k = x, \tb = k\}}) = \sum_{k = 1}^∞ \sum_{x \in B} α^k g(x) P_z(Z_k = x, \tb = k)\\ &= \sum_{k = 1}^∞ \sum_{x \in B} \sum_{y \in S} α^k g(x) P_z(Z_k = x, \tb = k \mid \color{red}{Z_1} = y) P_z(Z_1 = y).\tag{1} \end{align*} Note that $P_z(Z_1 = y) = p_{zy}$. For $y \in B$,$$ P_z(Z_k = x, \tb = k \mid Z_1 = y) = \begin{cases} δ_{xy}; & k = 1\\ 0; & k \geqslant 2 \end{cases}, $$ and for $y \in S \setminus B$,$$ P_z(Z_k = x, \tb = k \mid Z_1 = y) = \begin{cases} 0; & k = 1\\ P_y(Z_{k - 1} = x, \tb = k - 1); & k \geqslant 2\\ \end{cases}, $$ thus\begin{gather*} \sum_{x \in B} \sum_{y \in S} α g(x) P_z(Z_1 = x, \tb = 1 \mid Z_1 = y) p_{zy}\\ = α \sum_{y \in B} p_{zy} g(y) = α \sum_{y \in B} p_{zy} E_y\left( α^\tb g(Z_\tb) \right), \end{gather*}\begin{align*} &\peq \sum_{k = 2}^∞ \sum_{x \in B} \sum_{y \in S} α^k g(x) P_z(Z_k = x, \tb = k \mid Z_1 = y) p_{zy}\\ &= \sum_{k = 2}^∞ \sum_{x \in B} \sum_{y \in S \setminus B} α^k g(x) P_y(Z_{k - 1} = x, \tb = k - 1) p_{zy}\\ &= α \sum_{y \in S \setminus B} p_{zy} \sum_{k = 2}^∞ \sum_{x \in B} α^{k - 1} g(x) P_y(Z_{k - 1} = x, \tb = k - 1)\\ &= α \sum_{y \in S \setminus B} p_{zy} \sum_{k = 1}^∞ \sum_{x \in B} α^k g(x) P_y(Z_k = x, \tb = k)\\ &= α \sum_{y \in S \setminus B} p_{zy} \sum_{k = 1}^∞ \sum_{x \in B} E_y(α^k g(x) I_{\{Z_k = x, \tb = k\}})\\ &= α \sum_{y \in S \setminus B} p_{zy} \sum_{k = 1}^∞ E_y(α^k g(Z_k) I_{\{\tb = k\}}) = α \sum_{y \in S \setminus B} p_{zy} E_y\left( α^\tb g(Z_\tb) \right) \end{align*} and\begin{align*} (1) &= \sum_{x \in B} \sum_{y \in S} α g(x) P_z(Z_1 = x, \tb = 1 \mid Z_1 = y) p_{zy}\\ &\peq + \sum_{k = 2}^∞ \sum_{x \in B} \sum_{y \in S} α^k g(x) P_z(Z_k = x, \tb = k \mid Z_1 = y) p_{zy}\\ &= α \sum_{y \in B} p_{zy} E_y\left( α^\tb g(Z_\tb) \right) + α \sum_{y \in S \setminus B} p_{zy} E_y\left( α^\tb g(Z_\tb) \right)\\ &= α \sum_{y \in S} p_{zy} E_y\left( α^\tb g(Z_\tb) \right). \end{align*}