Exponential family of distributions and conditioning

142 Views Asked by At

The standard definition of the exponential family of distributions is that the probability density function of a random variable $X$ can be expressed as $$ p(x|\theta) = h(x) \exp\left\{ \theta' T(x) - A(\theta)\right\},$$ where $\theta$ are the natural parameters, $T(x)$ the sufficient statistics, $h(x)$ the base measure, and $A(\theta)$ the log-partition function.

Suppose now that the random variable $X|\{Y=y\}$ is in the exponential family (with natural parameter $\theta$); in general, because of conditioning on $\{Y = y\}$, the log partition function should depend on the value $y$: $$ p(x|y,\theta) = h(x,y) \exp\{\theta'T(x,y) - A(\theta,y)\}.$$ This seems to indicate that the prior distribution for the parameters will also be a function of $y$: $$ p(\theta|\chi,\nu) = g(\chi,\nu) \exp\left\{ \theta' \chi - \nu A(\theta,y)\right\}$$

However, for many examples, there is a prior distribution that is independent of the conditioning variable; for instance, in the Bayesian regression problem $$ X = \theta Y + E,$$ where $E \sim N(0,1)$, we have that

$$p(x|y,\theta) = \underbrace{\frac{1}{\sqrt{2\pi}}\exp\bigg\{ -\frac{x^2}{2} \bigg\}}_{h(x,y)} \exp\bigg\{ \theta \underbrace{\frac{xy}{2}}_{T(x,y)} -\underbrace{\theta^2}_{a(\theta)} \frac{y^2}{2}\bigg\}$$ and we can find a conjugate prior distribution, $$p(\theta|\chi,\nu) = g(\chi,\nu) \exp\{\theta \chi - \nu a(\theta)\},$$ which is independent of $y$, because $A(\theta,y) = a(\theta) f(y)$.

Is this a general result? Under what conditions does this happen? Do distributions of this type have a special name? Any reflection is welcome!