Question about exponential family

37 Views Asked by At

If an n-dimensional model $S=\{p_{\theta/ \theta\in\Theta}\}$ can be expressed in terms of the functions $\{C,F_1,..,F_n\}$ on a sample space $X$ and a function $\psi\in \Theta$(parameter space) as $$p(x;\theta)=exp[C(x)+\sum_{i=1}^{n}\theta^{i}F_{i}(x)-\psi(\theta)]$$ then we say that $S$ is an exponential family.

My question is , it is written that $\theta\mapsto p_{\theta}$ is one-to-one if and only if the $(n+1)$ functions $\{F_1,...,F_n,1\}$ are linearly independent. But I am not getting this line. Can someone explain it or give some hint. Thanks.

1

There are 1 best solutions below

0
On

Explanation of the definition

$\{F_1, \ldots, F_n, 1\}$ are linearly independent if there do not exist constants $c_1, \ldots, c_{n+1}$ not all zero such that $$c_1 F_1(x) + \cdots + c_n F_n(x) + c_{n+1} = 0 \quad \text{(for all $x$).}$$

What this has to do with $\theta \mapsto p_\theta$ being one-to-one.

Suppose $\theta_1$ and $\theta_2$ are parameters in $\Theta$ such that $p_{\theta_1} = p_{\theta_2}$, i.e. $$C(x) + \sum_{i=1}^n \theta^i_1 F_i(x) - \psi(\theta_1) = C(x) + \sum_{i=1}^n \theta^i_2 F_i(x) - \psi(\theta_2).$$ Subtracting the two equations leads to $$\sum_{i=1}^n (\theta_2^i - \theta_1^i) F_i(x) + (\psi(\theta_2) - \psi(\theta_1)) = 0.$$ Because $\theta_1 \ne \theta_2$, at least one of the $\theta_2^i-\theta^i-1$ is nonzero, so this shows $\{F_1, \ldots, F_n, 1\}$ is linearly dependent.