Constructing a Borel-measurable function from a functional inequality

160 Views Asked by At

Let $(X, \mathcal{X})$ and $(A, \mathcal{A})$ be standard Borel spaces, and let $q(\cdot | \cdot)$ be a substochastic kernel with source $(X\times A, \mathcal{X}\otimes\mathcal{A})$ and target $(X, \mathcal{X})$. The latter means $q: X \times A \times \mathcal{X} \rightarrow [0,1]$ is such that (i) $(x,a) \mapsto q(B | x, a)$ is $(\mathcal{X}\otimes\mathcal{A})$-measurable for each $B \in \mathcal{X}$ and (ii) $B \mapsto q(B | x, a)$ is a sub-probability measure on $(X, \mathcal{X})$ for each $(x, a) \in X \times A$.

Suppose the bounded function $f: X \rightarrow [1, \infty)$ is upper semi-analytic, i.e. the set $\{x \in X: f(x) > \lambda\}$ is analytic for every $\lambda \geq 0$, and satisfies

$$ f(x) \geq 1 + \int_X f(y)q(dy | x, a) \quad \text{for all} \ \ (x, a) \in X \times A. \qquad \text{(1)} $$

Question: Does there also exist a bounded Borel-measurable function that satisfies (1)?

Context: This question came up in the context of Markov decision processes; see e.g. this book. There, $X$ is the set of states of a controlled random process, and $A$ is the set of available actions. If action $a$ is taken when the system is in state $x$, a reward is earned and the next state belongs to $B \in \mathcal{X}$ with probability $q(B | x, a)$ or the process terminates with probability $1-q(X | x, a)$.

1

There are 1 best solutions below

1
On BEST ANSWER

Given an integral inequality of the kind as in OP, the answer is yes at least in case whe there exists a probability measure $\mu$ such that $p(x,a) \ll \mu$ for all $x,a$. Then one can modify $f$ on a $\mu$-null set $N$ to make it Borel-measurable. Then the RHS does not change, and for all $x\notin N$ the inequality still holds. On $N$ we can define $f$ to be the RHS (yet again, it does not change the RHS), which is a Borel function due to the fact that $f$ and $p$ are Borel.