exp concavity of hellinger loss

72 Views Asked by At

Let $\mathcal{Y} = \mathcal{D} = [0,1]$ and consider the Hellinger loss \begin{equation} \ell(z, y) = \frac12 \left((\sqrt{z} - \sqrt{y})^2 + (\sqrt{1 - z} - \sqrt{1-y})^2 \right) \end{equation} Determine the values of $\eta$ for which the function $F(z) = e^{\eta\ell(z, l)}$ is concave.

Since this is a function in $1$ variable I can take the second partial derivative with respect to $z$ and find $\eta$ such that this second derivative is negative, but it ends up being intractable.

1

There are 1 best solutions below

3
On

I am assuming $l \in [0,1]$. Break the functions up: let $$f(z) = e^{\eta z/2} \qquad g(z) = (\sqrt{z} - \sqrt{l})^2 + (\sqrt{1-z} - \sqrt{1-l})^2 \qquad h(z) = f(g(z)).$$ Use the chain rule to compute $$h''(z) = f'(g(z))g''(z) + f''(g(z))(g'(z))^2.$$ We have 2 observations that you should be able to verify:

  1. $g(z)$ is convex in $[0,1]$.
  2. $f(z)$ is convex.

Using these observations, conclude $h$ is convex if $\eta > 0$. (I have to check the concave case.) Looking at the functions in a graphing calculator of your choice (desmos) might help you.