Consider a finite set $\mathcal{S}.$ Let $P$ and $Q$ be two transition probability matrices on $\mathcal{S} \times \mathcal{S}$, with each row summing to $1.$ Given $i,j \in \mathcal{S}$, let $P(i,j)$ denote the $(i,j)$th entry of $P,$ and let $P(i,\cdot)$ denote the $i$th row of $P.$ Given an integer $d \geq 1,$ let $P^d$ denote the transition probability matrix obtained by multiplying $P$ with itself $d$ times. For any $i \in \mathcal{S}$ and $d \geq 1,$ let $$ D_{\text{KL}}(P^d(i, \cdot) \parallel Q^d(i, \cdot)) = \sum_{j \in \mathcal{S}} P^d(i, j) \log \frac{P^d(i, j)}{Q^d(i, j)} $$ denote the Kullback—Leibler divergence between the $i$th rows of $P^d$ and $Q^d.$ If $P$ is irreducible, then there exists a unique probability distribution $\mu$ on $\mathcal{S}$ such that $\mu = \mu \cdot P^d$ for all $d \geq 1.$ Consider the conditional Kullback—Leibler divergence $$ D_{\text{KL}}(P^d \parallel Q^d \mid \mu) = \sum_{i \in \mathcal{S}} \mu(i)\, D_{\text{KL}}(P^d(i, \cdot) \parallel Q^d(i, \cdot)). $$
Given any $d \geq 1$ and $i \in \mathcal{S}$, I am interested in obtaining an upper bound for $D_{\text{KL}}(P^d(i, \cdot) \parallel Q^d(i, \cdot))$ in terms of the conditional divergence $D_{\text{KL}}(P \parallel Q \mid \mu).$ Notice that the latter does not have $d$ featuring in it.
A Naive Bound:
The following result is easy to show using the log-sum inequality. $$ \begin{align*} \forall d\geq 1, \qquad D_{\text{KL}}(P^d \parallel Q^d \mid \mu) \leq d\, D_{\text{KL}}(P \parallel Q \mid \mu). \end{align*} $$ Using the above result, we then note that $$ \begin{align*} D_{\text{KL}}(P^d (i, \cdot) \parallel Q^d(i, \cdot)) & = \frac{1}{\mu(i)} \cdot \mu(i)\cdot D_{\text{KL}}(P^d (i, \cdot) \parallel Q^d(i, \cdot)) \\ &\leq \frac{1}{\mu(i)} \cdot D_{\text{KL}}(P^d \parallel Q^d \mid \mu) \\ &\leq \frac{d}{\mu(i)} \, D_{\text{KL}}(P \parallel Q \mid \mu) \\ &\leq \frac{d}{\mu_{\text{min}}} \, D_{\text{KL}}(P \parallel Q \mid \mu), \end{align*} $$ where $\mu_{\text{min}} = \min_i \mu(i)>0.$
I am looking for a bound that does not have $\mu_{\text{min}}$ in the denominator. Any suggestions/leads will be greatly appreciated. Many thanks in advance.