Existence of Frechet Derivative of Log-Normalizer for Exponential Family Type Distribution

90 Views Asked by At

Let's say we have a density function that generalizes the exponential family slightly \begin{align} p(t)=q(t)\exp(f(t)-A(f)) \end{align} where if $f(t)=\theta^T \phi(t)$ we recover the exponential family. Assume that $f\in L^\infty$, so that $A:L^\infty\rightarrow\mathbb{R}$. We want to check if $A(f)=\log\int q(t)\exp(f(t))dt$ is Frechet differentiable. We can start by computing the Gateaux differential \begin{align} \frac{d}{d\epsilon}\biggr\vert_{\epsilon=0} A(f+\epsilon g)&=\frac{\int q(t)\exp(f(t)+\epsilon g(t))g(t)dt}{\int q(t)\exp(f(t))dt}\biggr\vert_{\epsilon=0}\\ &=\mathbb{E} g(T) \end{align} Now we would like to show \begin{align} A(f+g)=A(f)+\frac{d}{d\epsilon}\biggr\vert_{\epsilon=0} A(f+\epsilon g)+o(\Vert g\Vert) \end{align} in order to show existence of the Frechet derivative (if it does in fact exist). We start with \begin{align} A(f+g)-A(f)&=\log \frac{\int q(t)\exp(f(t)+g(t))dt}{\int q(t)\exp(f(t))dt}\\ &=\log \mathbb{E} \exp (g(T)) \end{align} We then want to show that \begin{align} \lim_{g\rightarrow 0}\frac{\log \mathbb{E}\exp(g(T))-\mathbb{E}g(T)}{\Vert g\Vert_\infty}=0 \end{align} I'm not sure how to show that either this holds or doesn't hold.