Let
- $(E,\mathcal E,\lambda)$ be a measure space
- $p:E\to[0,\infty)$ be $\mathcal E$-measurable with $$\int p\:{\rm d}\lambda=1$$
- $\mu:=p\lambda$
- $f\in\mathcal L^1(\mu)$
Now let $$\Phi(q):=\int_{\left\{\:q\:>\:0\:\right\}}\frac{(pf)^2}q\:{\rm d}\lambda$$ for $q\in\mathcal L^1(\lambda)$ with $q\ge0$. Are we able to rigorously prove that the Fréchet derivative of $\Phi$ exists and is equal to $-\frac{(pf)^2}{q^2}$?
My first problem is a formal one: A mapping can only be Fréchet derivative on an open subset of a Banach space. We would clearly want to consider the Banach space $L^1(\lambda)$, but how do we need to define the open suset $\Omega$?
If this question is clarified, we should need to show $$\frac{\left|\Phi(q+h)-\Phi(q)+\frac{(pf)^2}{h^2}\right|}{\left\|h\right\|_{L^1(\lambda)}}\xrightarrow{h\to0}0\tag1$$ for all $q\in\Omega$. How can we do this? (If at all.)
Please take note of my related question: Minimize $q\mapsto\int\frac{(pf)^2}q\:{\rm d}\lambda$ subject to $\int q\:{\rm }\lambda=1$ using the method of Lagrange multipliers.
The following is not rigorous, but provides a first insight into why this should be the correct derivative:
Note that for $g(x) = \frac{1}{x}$ you obtain (Taylor series) $$ g(x+h) = g(x) + g'(x)h = \frac{1}{x} - \frac{h}{x^2} + o(|h|) $$ Therefore, $\frac{1}{q+h}$ has the following linear approximation (for small $h$): $$ \frac{1}{q+h} \approx \frac{1}{q} - \frac{h}{q^2}, $$ which implies $$ \Phi(q+h) = \int \frac{(pf)^2}{q+h}\, \mathrm d\lambda \approx \int \frac{(pf)^2}{q}\, \mathrm d\lambda - \int \frac{(pf)^2}{q^2}\, h\, \mathrm d\lambda = \Phi(q) + \Big\langle -\frac{(pf)^2}{q^2}\, ,\, h \Big\rangle_{L^2(\lambda)} $$ Since we are considering the best linear approximation, the linear map $$ A\colon h\mapsto \Big\langle -\frac{(pf)^2}{q^2}\, ,\, h \Big\rangle_{L^2(\lambda)} $$ fulfills the Frechet derivative property: $$ \frac{\left|\Phi(q+h)-\Phi(q)-Ah\right|}{\left\|h\right\|}\xrightarrow{h\to 0} 0. $$ (Note that in your equation (1) you are adding real numbers and functions, which can not be correct - you really need to define the linear map $h\mapsto Ah$ in a proper way).
You obviously need to fill in some blanks to make this rigorous, in particular which spaces you are working in ($L^1$ or $L^2$). You should also verify that your map $\Phi$ is well-defined, meaning that the integral exists for $f\in\mathcal L^1(\mu)$ and any $q$. I hope this helps..