
Here is the definition of Relative Entropy Now I am only interested in the simplest condition that the index set is finite and discrete, as the naive probability distribution vectors.
Now if the distribution vector $p_3=a p_1+(1-a) p_2$, $q_3=a q_1+(1-a)q_2$ for $0<a<1$ how to prove that $H(p1|q1)<H(p3|q3)<H(p2|q2)$?
The right hand side can be proved with the joint convexity of relative entropy, that is,
the left hand side is not valid as I see. For example, suppose
P_1=Q_2=(0.3,0.7),P_2=Q_1={0.7,0.3}, anda=0.5, then the middle term vanishes while the left side is positive.