I'm trying to derive equations from Matthew Beal's Thesis, Variational Algorithms for Approximate Bayesian Inference pg.55, but I'm stuck on one of the equations (well I'm stuck on a lot of equations actually). This is the equation I would like help on:
$$\frac{\delta}{\delta q_x(x)} \int dxq_x(x)\ln \frac{P(x,y)}{q_x(x)}\tag{1}$$ The equation is suppose to equate to $$\ln P(x,y)-\ln q_x(x)-1\tag{2}$$ Of course, (1) can be expanded to $$\frac{\delta}{\delta q_x(x)} \int dxq_x(x)(\ln P(x,y)-\ln q_x(x))$$ and then once more into $$\frac{\delta}{\delta q_x(x)} \int dxq_x(x)\ln P(x,y)-\frac{\delta}{\delta q_x(x)} \int dxq_x(x)\ln q_x(x)$$ I'm guessing $$\frac{\delta}{\delta q_x(x)} \int dxq_x(x)\ln P(x,y)=\ln P(x,y)$$ and $$-\frac{\delta}{\delta q_x(x)} \int dxq_x(x)\ln q_x(x)=-\ln q_x(x)-1$$ But why? Any help would be appreciated. The equations were also simplified from their original complexity.
I'm a little bit rusty in variational calculus, but i think it should be the following way:
Remember $\frac{\delta q(x)}{\delta q(y)}=\delta(x-y)$ (1) and (the delta at right side is a delta distribution) and the chain rule $\frac{\delta f(q(x))}{\delta q(y)}=\delta(x-y)\partial_q f$ (2). There is also a product rule for functional derivatives $\frac{\delta [a(q(x))p(q(x))]}{\delta q(x)}=\frac{\delta a(q(x))}{\delta q(x)}p(q(x))+\frac{\delta p(q(x))}{\delta q(x)}a(q(x))$ (3)
Now apply this rule to your equations and perform the trivial integrals. Can you take it from here?
For example:
$$ \frac{\delta}{\delta q(x')}\int{dxq(x)\log(q(x))}\underbrace{=}_{3}\int dx \frac{\delta q(x)}{\delta q(x')}\log(q(x))+\int dx \frac{\delta log(q(x))}{\delta q(x')}q(x)\\ \underbrace{=}_{(1),(2)}\int{\delta}(x-x')\log(q(x))+\int{\delta}(x-x')\frac{q(x)}{q(x)}=\log(q(x'))+1 $$