Convexity of KL-Divergence

2.7k Views Asked by At

If $0\leq \lambda \leq 1$, and we have probability mass functions $p_1,p_2,q_1,q_2$, then using the log sum inequality we can show that

${\displaystyle D_{\mathrm {KL} }(\lambda p_{1}+(1-\lambda )p_{2}\|\lambda q_{1}+(1-\lambda )q_{2})\leq \lambda D_{\mathrm {KL} }(p_{1}\|q_{1})+(1-\lambda )D_{\mathrm {KL} }(p_{2}\|q_{2})}$

How about the case where $p_1,p_2,q_1,q_2$ are pdfs (continuous random variables)?

Do we have a similar thing for continuous random variables?