Problem: I have an integral of the form
$$\int_{-\infty}^{\infty}p(x)\text{log}\frac{p(x)}{g(x,v)}dx$$
and I want to find the $v$ that minimizes it.
In particular, $p(x)$ and $g(x,v)$ are PDFs, where
$$p(x)=\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}(x-1)^2}$$ and $$g(x,v)=\frac{1}{2\sqrt{2 \pi}}e^{-\frac{1}{2}(x-2)^2}+\frac{1}{2\sqrt{2 \pi}}e^{-\frac{1}{2}(x-v)^2},$$
and of course $\int p(x) dx=1$ and $\int g(x,v) dx=1$.
I know that the solution is around $v=0.55$ from numerical approximations but I can't seem to get to that answer analytically.
I'm trying to approach this problem from a variational calculus perspective. In this problem, we can only vary $g(x,v)$ and in fact we can really only vary the value of $v$. So I set up a functional with the constraint on $g$:
$$J[g]=\int_{-\infty}^{\infty}p(x)\text{log}\frac{p(x)}{g(x,v)}+\lambda g(x,v)dx,$$ and accordingly the Euler-Lagrange equation for $g$ is:
$$\frac{\delta J}{\delta g}=\frac{-p(x)}{g(x,v)}+\lambda=0$$
The first equation suggests (correctly, generally speaking) that $p=g$. However this doesn't help me much as in our example, $g$ cannot take on the form $p$ for any $v$.
My next attempt was to directly take $\frac{\delta J}{\delta g}=\frac{\partial}{\partial v}$ since we can only change $v$. Then:
$$\frac{\delta J}{\delta g}=\frac{\partial}{\partial v}=\frac{p(x)g_v(x,v)}{g(x,v)}+\lambda g_v(x,v)=0$$
However this just turns out to be $p(x)g_v(x,v)=g(x,v)\lambda g_v(x,v)$ which implies the same thing, that $p=g$.
I don't really know how to solve this problem. Any help would be appreciated!
Note: beyond this one problem, it would be ideal to have a solution for a general $$p(x)=\frac{1}{n}\sum_{x_i \in X}\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}(x-x_i)^2}$$ and $$g(x,v)=\frac{1}{m}\sum_{x_i \in W}\left(\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}(x-x_i)^2}\right)+\frac{1}{\sqrt{2 \pi}}e^{-\frac{1}{2}(x-v)^2}$$
Elaborating on my comment, differentiating with respect to $v$ and setting equal to $0$ gives $$ -\int_{-\infty}^\infty p(x)\frac{\partial_v g(x,v)}{g(x,v)}dx =\int_{-\infty}^\infty \frac{x-v}{\sqrt{2\pi}}\frac{e^{-(x-x_1)^2/2}e^{-(x-v)^2/2}}{e^{-(x-x_2)^2/2} + e^{-(x-v)^2/2}}dx = 0 $$ This is a pretty spicy integral, but the shift $x\rightarrow u + (v+x_2)/2$ can get it into a nicer form. Making this substitution and defining $a = x_1-x_2$ and $b = (v-x_2)/2$ gives $$ \frac{e^{-(a-b)^2/2}}{2\sqrt{2\pi}}\int_{-\infty}^\infty(u-b)e^{-u^2/2}\operatorname{sech}(b u)e^{a u}du = 0. $$ Since the constant term outside the integral is always nonzero, this condition requires that the integral itself be zero.
That's about the best I could do, as the integral doesn't show up in any table I could find and it doesn't convert to a differential equation easily. However, this integral equation does implicitly define a function. That is, there is some single variable function $f$ such that $$ \int_{-\infty}^\infty[u-f(a)]e^{-u^2/2}\operatorname{sech}[f(a) u]e^{a u}du = 0 $$ is satisfied for all $a$. There may be some trickery using implicit differentiation and integration by parts to turn that integral into a differential equation for $f$, but I was unable to find any. At any rate, in terms of this function, we have $$ v = x_2 + f(x_1-x_2). $$