I am trying to re-derive the variational lower bound for the binary logistic regression that is obtained in the paper by Saul & Jordan 1999 and it is given in equation 22 $$\langle \ln(1+e^z)\rangle\leq\frac{1}{2}\xi^2\langle\delta z^2\rangle+\ln\Big(1+e^{\langle z\rangle+(1-2\xi)\langle\delta z^2\rangle/2)}\Big)$$
Could anybody suggest how I can get from equation 21 to 22?
In a similar topic, in a paper by Knowles and Minka 2011 they found another bound which is given in section 5.1 (in the third paragraph) which is again not clear how it is computed. Any suggestion or thought? Thanks
Using the moment generating function of the normal distribution, given by $\langle e^{tz}\rangle = e^{t\langle z \rangle + \tfrac{1}{2}\langle \delta z^2\rangle t^2}$, and the linearity of expectation we can write $$ \langle e^{-\xi z} + e^{(1-\xi)z} \rangle = \langle e^{-\xi z} \rangle + \langle e^{(1-\xi)z} \rangle = e^{-\xi\langle z \rangle + \tfrac{1}{2}\langle \delta z^2\rangle \xi^2} + e^{(1-\xi)\langle z \rangle + \tfrac{1}{2}\langle \delta z^2\rangle (1-\xi)^2} \\ = e^{-\xi\langle z \rangle + \tfrac{1}{2}\langle \delta z^2\rangle \xi^2}(1 + e^{\langle z \rangle + \tfrac{1}{2}\langle \delta z^2\rangle (1-2\xi)}).$$ From here on you should be able to work out the term $\xi\langle z\rangle + \ln\langle e^{-\xi z} + e^{(1-\xi)z} \rangle$.