Define two slightly different probability distributions: $$ p_k = \frac{ \exp \left[ - \eta L_k \right] }{ \sum\limits_{i=1}^K \exp \left[ - \eta L_{i} \right] }, \quad \tilde{p}_k = \frac{ \exp \left[ - \eta ( L_k + \ell_k ) \right] }{ \sum\limits_{i=1}^K \exp \left[ - \eta (L_{i} + \ell_{i} )\right] } $$ in which $L_k$ and $\ell_k$ are chosen randomly from some unknown distribution. Each of the probabilities $p$ and $\tilde{p}$ is defined over $K$ actions (e.g. $K = 2$ in Binomial case). Define a shorthand notation: $$ p = (p_1, p_2, p_3, ..., p_K) $$ Similarly define $\tilde{p}$ which is a slightly modified version of $p$.
Suppose we sample $a$ according to $p$: $$ \mathbb{P}(a=k) = p_k $$
Any ideas if we can find a non-trivial upperbound on $\mathbb{E}\left[ \frac{\tilde{p}_k}{p_k} a \right]$ which is of the form $D(p, \tilde{p})+ \mathbb{E}[a]$, where $D(p, \tilde{p})$ is a divergence measure (that we get to choose) between $p$ and $\tilde{p}$?
Trivially when $p = \tilde{p}$ then $D(p, \tilde{p}) = 0$ and $\mathbb{E}\left[ \frac{\tilde{p}_k}{p_k} a \right] = \mathbb{E}\left[ a \right]$.