Inequality related to cross-enropy

26 Views Asked by At

Suppose, $S: x \mapsto \mathbb{R}^1$ and $f(\cdot;p)$ is some parametric family of pdfs. Moreover, $\sum_{x}1 \{S(X) \geq \gamma \} f(x;u)\log f(x;v) \geq \sum_{x}1\{S(X)\geq \gamma\}f(x;u)\log f(x;u)$. It implies that, $$\sum_{x}1\{S(X)\geq \gamma\}f(x;v)\log f(x;v) \geq \sum_{x}1\{S(X)\geq \gamma\}f(x;v)\log f(x;u).$$ Based on the last equation, I would like to prove that, for any $f(x;k)$: $$\sum_{x} 1\{S(X)\geq \gamma\} f(x;u) \log f(x;k) \leq \sum_{x} 1\{S(X)\geq \gamma\} f(x;u) \log f(x;v), $$ The following inequality $$\sum_{x} 1\{S(X)\geq \gamma\}f(x;v) \log f(x;v) \geq \sum_{x} 1\{S(X)\geq \gamma\} f(x;v) \log f(x;k) ,$$ Holds.