Minimize $q\mapsto\int\frac{(pf)^2}q\:{\rm d}\lambda$ subject to $\int q\:{\rm }\lambda=1$ using the method of Lagrange multipliers

Question

Let

• $(E,\mathcal E,\lambda)$ be a measure space
• $p:E\to[0,\infty)$ be $\mathcal E$-measurable with $$\int p\:{\rm d}\lambda=1$$
• $\mu:=p\lambda$
• $f\in\mathcal L^1(\mu)$

I want to minimize $$\Phi(q):=\int_{\left\{\:q\:>\:0\:\right\}}\frac{(pf)^2}q\:{\rm d}\lambda$$ over all $\mathcal E$-measurable $q:E\to[0,\infty)$ subject to $$\int q\:{\rm d}\lambda=1.\tag1$$ I already know that the solution is proportional to $p|f|$, but I want to verify this rigorously.

I want to use he method of Lagrange multipliers. We should be able to rephrase the problem in the following way: We want to minimize a functional on a Banach space subject to the condition that the norm of the candidate is $1$. We would clearly take the Banach space $\mathcal L^1(\mu)$ (note that $(1)$ is noting else than the norm of $q$ in this space).

How do we need to proceed in detail?

It's clear to me that it's sufficient to find a stationary point of the Lagrange function. It's then easy to show that the resulting candidate solution is a minimum (using the Cauchy-Schwarz inequality).

Please take note of my related question: How can we compute the Fréchet derivative of $q\mapsto\int\frac{(pf)^2}q\:{\rm d}\lambda$?.

Answer 1

Here is a problem that can be solved with solution proportional to $|p(x)f(x)|$.

Problem

Given:

• $(E, \mathcal{E}, \lambda)$
• Measurable functions $p:E\rightarrow [0,\infty)$, $f:E\rightarrow\mathbb{R}$
• $\int_E p(x)d\lambda = 1$
• $0< \int_E |f(x)p(x)|d\lambda < \infty$.
• $p(x)f(x)\neq 0$ for all $x \in E$.

We want to find a measurable function $q:E\rightarrow[0,\infty)$ to minimize $\int_E \frac{(p(x)f(x))^2}{q(x)}d\lambda $ subject to:

1. $\int_E q(x)d\lambda = 1$

2. $q(x)>0$ for all $x \in E$.

Minimizer

Define the measurable function $q:E\rightarrow [0,\infty)$ by $$ q(x) = \frac{1}{c}|p(x)f(x)| \quad \forall x \in E $$ where $c$ is defined $$ c = \int_E |p(x)f(x)|d\lambda $$

Clearly this function $q(x)$ satisfies the desired constraints 1 and 2. It remains to prove it minimizes the objective over all other measurable functions $r:E\rightarrow [0,\infty)$ that satisfy constraints 1 and 2.

Optimality proof

Fix a measurable function $r:E\rightarrow [0,\infty)$ that satisfies constraints 1 and 2, so that $\int_E r(x)dx = 1$ and $r(x)>0$ for all $x \in E$. Fix $x \in E$. Note that $q(x)$ defined above is chosen as the value $q \in (0,\infty)$ that minimizes the expression $$ \frac{(p(x)f(x))^2}{q} + c^2q $$ where this expression is convex in $q$ and has a unique minimizer in $(0,\infty)$ (recall that $(p(x)f(x))^2>0$). Since $r(x)>0$ we have $$ \frac{(p(x)f(x))^2}{q(x)} + c^2q(x) \leq \frac{(p(x)f(x))^2}{r(x)} + c^2r(x) \quad \forall x \in E$$ Integrating the above inequality gives $$ \int_E \frac{(p(x)f(x))^2}{q(x)}d\lambda + c^2 \underbrace{\int_E q(x)d\lambda}_{1} \leq \int_E \frac{(p(x)f(x))^2}{r(x)}d\lambda + c^2\underbrace{\int_E r(x)d\lambda}_{1}$$ where the underbrace equalities hold because both $q$ and $r$ satisfy constraint 1. Canceling common terms yields $$ \int_E \frac{(p(x)f(x))^2}{q(x)}d\lambda \leq \int_E \frac{(p(x)f(x))^2}{r(x)}d\lambda$$ $\Box$

Answer 2

Counter-example

Here is a counter-example to show the solution will not necessarily be proportional to $|p(x)f(x)|$ over all $x \in E$, and the problem can have degenerate cases:

Define:

• $E=[0,1]$ with the usual Lebesgue measure.

• $p(x) = 1, f(x)=x$ for all $x \in [0,1]$.

For each $d \in (0,1]$ define

$$q_d(x) = \left\{ \begin{array}{ll} \frac{2x}{d^2} &\mbox{ if $x\in [0,d]$} \\ 0 & \mbox{ otherwise} \end{array} \right.$$ Then $\int_0^1 q_d(x)dx = 1$ for all $d \in (0,1]$ and $$ \int_{x:q(x)>0} \frac{(p(x)f(x))^2}{q(x)}dx = \frac{d^2}{2}\int_0^dx dx = \frac{d^4}{4}$$

Now the function $q_1(x)$ is proportional to $|p(x)f(x)|$ over all $x \in [0,1]$, but this has objective function $\frac{d^4}{4}|_{d=1}= 1/4$. We can do better by pushing $d\rightarrow 0$ to get an infimum objective value of $0$. This is a degenerate case when there is no minimizer but we can find a sequence of functions that satisfy the constraints and that have objective values that converge to the infimum of 0.

---

General case

You can repeat the example to solve almost all general cases this way: Fix $(E,\mathcal{E}, \lambda)$ and fix $p:E\rightarrow [0,\infty)$, $f:E\rightarrow\mathbb{R}$ and suppose that for all positive integers $n$ there is a measurable set $B_n \subseteq E$ such that $$ 0<\int_{B_n} |p(x)f(x)|d\lambda \leq 1/n$$ Define $$c_n = \int_{B_n} |p(x)f(x)|d\lambda \quad \forall n \in \{1, 2, 3, ..\}$$ and note that $0<c_n\leq 1/n$. For each $n \in \{1, 2, 3, ...\}$ define $q_n:E\rightarrow[0,\infty)$ by $$ q_n(x) = \left\{ \begin{array}{ll} \frac{|p(x)f(x)|}{c_n} &\mbox{ if $x\in B_n$} \\ 0 & \mbox{ otherwise} \end{array} \right.$$ Then $\int q_n(x)d\lambda = \frac{1}{c_n}\int_{B_n} |p(x)f(x)|d\lambda=1$ for all $n \in \{1, 2, 3, ...\}$ but $$ \int_{x:q_n(x)>0}\frac{(p(x)f(x))^2}{q_n(x)} d\lambda = c_n\int_{B_n}|p(x)f(x)|d\lambda = c_n^2\rightarrow 0$$ So the infimum objective value is 0.

If we assume that $\lambda(\{x \in E : p(x)f(x)=0\}) = 0$ then it can be shown that it is impossible to achieve an objective value of 0. Thus, this situation is degenerate: There is no minimizer, but there is an infinite sequence of functions that satisfy the constraints and that have objective function that converges to the infimum of 0.

On the other hand, if we can find a measurable set $B\subseteq E$ such that $0<\lambda(B) < \infty$ and $p(x)f(x)=0$ for all $x \in B$, then we can easily achieve the optimal objective value of $0$ with $$ q(x) = \left\{ \begin{array}{ll} \frac{1}{\lambda(B)} &\mbox{ if $x \in B$} \\ 0 & \mbox{ otherwise} \end{array} \right.$$