The beta distribution with parameters $\alpha>0$ and $\beta>0$ has density$$f(x|\alpha,\beta)=\frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}x^{\alpha-1}(1-x)^{\beta-1},0<x<1$$ Suppose $X_1,...,X_n$ are independently and identically distributed from a beta distribution.
- Determine a minimal sufficient statistic (for the family of joint distributions) if $\alpha$ and $\beta$ very freely
- Determine a minimal sufficient statistic if $\alpha=2\beta$
- Determine a minimal sufficient statistic if $\alpha=\beta^2$
The above is the setup of our homework question for the statistical inference course. For (a),
$$\prod f(y_i|\alpha,\beta)=\frac{1}{B^n(\alpha,\beta)}\exp\{(\alpha-1)\sum\log y_i+(\beta-1)\sum\log(1-y_i)\}$$
$$\frac{\prod f(x_i|\alpha,\beta)}{\prod f(y_i|\alpha,\beta)}=\exp\{(\alpha-1)\sum(\log x_i-\log y_i)+(\beta-1)\sum(\log(1-x_i)-\log(1-y_i))\}$$
If $\sum\log x_i=\sum\log y_i, \sum \log(1-x_i)=\sum\log(1-y_i)$, the ratio is constant, therefore by Lehmann-Scheffe theorem, $$\{T_1=\sum\log x_i,T_2=\sum\log(1-x_i)\}$$ is a minimal sufficient statistic for $(\alpha,\beta)$.
The above is how I approached this question. I first used the factorization criterion to find the sufficient statistics and prove it is also minimal sufficient. However, coming to the second part, I tried to approach the question the same way:
$$f(x|\beta)=\frac{\Gamma(3\beta)}{\Gamma(2\beta)\Gamma(\beta)}x^{2\beta-1}(1-x)^{\beta-1}$$
$$=\frac{\Gamma(3\beta)}{\Gamma(2\beta)\Gamma(\beta)}\frac{1}{x(1-x)}\exp\{2\beta\log x+\beta\log (1-x)\}$$
$$=\frac{\Gamma(3\beta)}{\Gamma(2\beta)\Gamma(\beta)}\frac{1}{x(1-x)}\exp\{\beta(2\log x+\log(1-x))\}$$
$$=\frac{\Gamma(3\beta)}{\Gamma(2\beta)\Gamma(\beta)}\frac{1}{x(1-x)}\exp\{\beta\log x^2(1-x)\}$$
$$T(X)=\sum\log x_i^2(1-x_i)=2T_1+T_2$$
However, what I have done so far, in my view, is merely showing that the statistic is sufficient but how to show it is also the minimal sufficient statistic I do not know. I tried to find the answer online and found This material from Purdue's website. The question looks exactly the same, and the procedure for the first two questions seem not so different from what I have done. I am perplexed about how come the answer directly jump to that the statistic is also minimal sufficient? Did I miss any theorem? Also does $T(X)=\sum\log x_i^2(1-x_i)=2T_1+T_2$ actually carry with some implications?
Also, when approaching this question I have a more general and vague question that I want to ask. It appears that other than making a exponential family in an exponential form, Fisher-Neyman Factorization Theorem is enough to establish the sufficient statistics, an it is usually in the product form: here is just an example for your reference . It always appear that the product can be transformed to summation through exponential and logarithm somehow. But when I studied the concepts in statistics I always struggled to make distinctions between the distribution of a statistic and the joint distribution of a set of random variables. I guess it is because most of the time I am dealing with normal distribution; therefore the summation of x and x-square just naturally appear in the joint distribution and I always thought when we take the product we obtain something connects with mean and variance. But these concepts turn out very confusing when I am dealing with other distributions.
For example, the beta distribution. So far from what I have learnt online, both $\prod x_i$,$\prod (1-x_i)$ and $\sum\log x_n , \sum \log (1-x_n)$ seem both valid. However, when considering them as statistics I feel very hard to understand what we are trying to estimate from these statistics. Also it appear to me they are estimating totally different thing, how come they are both efficient? If they are both efficient there must be a one-to-one correspondence if I am not mistaken. How should I link the concept of sufficient statistics with through estimates I obtained from LME and other estimation techniques?
I hope my question make sense. Thank you so much for your time and patience!
$a)$ We have that $T(\mathbf{X})=\left(\sum_{i=1}^n\log(X_i),\sum_{i=1}^n\log(1-X_i)\right)$ is sufficient for the family $\{B(\alpha,\beta),\alpha>0,\beta>0\}$ since for all $\mathbf{x}\in\mathbb R^n$, $\alpha>0$, and $\beta>0$,
$$\begin{align*} f(\mathbf{x}\mid\alpha,\beta) &=\left[\frac{\Gamma\left(\alpha+\beta\right)}{\Gamma\left(\alpha\right)\left(\beta\right)}\right]^n\left(\prod_{i=1}^n x_i\right)^{\alpha-1}\left(\prod_{i=1}^n (1-x_i)\right)^{\beta-1}\prod_{i=1}^n\mathbb{1}_{[0,1]}(x_i)\\\\ &=\underbrace{\exp\left[\left(\alpha-1\right)\sum_{i=1}^n\log(x_i)+(\beta-1)\sum_{i=1}^n\log(1-x_i)+n\log\left(\frac{\Gamma\left(\alpha+\beta\right)}{\Gamma\left(\alpha\right)\left(\beta\right)}\right)\right]}_{g(T(\mathbf{x})\mid\alpha,\beta)}\underbrace{\mathbb{1}_{[0,1]}(x_{(1)})\mathbb{1}_{[0,1]}(x_{(n)})}_{h(\mathbf{x})} \end{align*}$$
We also have that $T(\mathbf{X})=\left(\sum_{i=1}^n\log(X_i),\sum_{i=1}^n\log(1-X_i)\right)$ is a minimal sufficient statistic since for any two possible sample points $\mathbf{x}$ and $\mathbf{y}$
$$\begin{align*} \frac{f\left(\mathbf{x}\mid\alpha,\beta\right)}{f\left(\mathbf{y}\mid\alpha,\beta\right)} &=\frac{\exp\left[\left(\alpha-1\right)\sum_{i=1}^n\log(x_i)+(\beta-1)\sum_{i=1}^n\log(1-x_i)+n\log\left(\frac{\Gamma\left(\alpha+\beta\right)}{\Gamma\left(\alpha\right)\left(\beta\right)}\right)\right]}{\exp\left[\left(\alpha-1\right)\sum_{i=1}^n\log(y_i)+(\beta-1)\sum_{i=1}^n\log(1-y_i)+n\log\left(\frac{\Gamma\left(\alpha+\beta\right)}{\Gamma\left(\alpha\right)\left(\beta\right)}\right)\right]}\\\\ &=\exp\left[\left(\alpha-1\right)\left[\sum_{i=1}^n\log(x_i)-\sum_{i=1}^n\log(y_i)\right]+(\beta-1)\left[\sum_{i=1}^n\log(1-x_i)-\sum_{i=1}^n\log(1-y_i)\right]\right] \end{align*}$$
does not depend on $\alpha$ or $\beta$ if and only if $T(\mathbf{x})=\left(\sum_{i=1}^n\log(x_i),\sum_{i=1}^n\log(1-x_i)\right)$ equals $T(\mathbf{y})=\left(\sum_{i=1}^n\log(y_i),\sum_{i=1}^n\log(1-y_i)\right)$
$b)$ We have that $T(\mathbf{X})=\sum_{i=1}^n\log\left(X_i^2(1-X_i)\right)$ is sufficient for the family $\{B(\alpha,\beta),\alpha=2\beta,\beta>0\}$ since for all $\mathbf{x}\in\mathbb R^n$ and $\beta>0$,
$$\begin{align*} f(\mathbf{x}\mid\alpha=2\beta,\beta) &=\left[\frac{\Gamma\left(2\beta+\beta\right)}{\Gamma\left(2\beta\right)\left(\beta\right)}\right]^n\left(\prod_{i=1}^n x_i\right)^{2\beta-1}\left(\prod_{i=1}^n (1-x_i)\right)^{\beta-1}\prod_{i=1}^n\mathbb{1}_{[0,1]}(x_i)\\\\ &=\exp\left[\left(2\beta-1\right)\sum_{i=1}^n\log(x_i)+(\beta-1)\sum_{i=1}^n\log(1-x_i)+n\log\left(\frac{\Gamma\left(3\beta\right)}{\Gamma\left(2\beta\right)\left(\beta\right)}\right)\right]\mathbb{1}_{[0,1]}(x_{(1)})\mathbb{1}_{[0,1]}(x_{(n)})\\\\ &=\exp\left[(\beta-1)\sum_{i=1}^n\log\left(x_i^2(1-x_i)\right)+\sum_{i=1}^n\log(x_i)+n\log\left(\frac{\Gamma\left(3\beta\right)}{\Gamma\left(2\beta\right)\left(\beta\right)}\right)\right]\mathbb{1}_{[0,1]}(x_{(1)})\mathbb{1}_{[0,1]}(x_{(n)})\\\\ &=\underbrace{\exp\left[(\beta-1)\sum_{i=1}^n\log\left(x_i^2(1-x_i)\right)+n\log\left(\frac{\Gamma\left(3\beta\right)}{\Gamma\left(2\beta\right)\left(\beta\right)}\right)\right]}_{g(T(\mathbf{x})\mid\alpha=2\beta,\beta)}\underbrace{\exp\left(\sum_{i=1}^n\log(x_i)\right)\mathbb{1}_{[0,1]}(x_{(1)})\mathbb{1}_{[0,1]}(x_{(n)})}_{h(\mathbf{x})}\\\\ \end{align*}$$
We also have that $T(\mathbf{X})=\sum_{i=1}^n\log\left(X_i^2(1-X_i)\right)$ is a minimal sufficient statistic since for any two possible sample points $\mathbf{x}$ and $\mathbf{y}$
$$\begin{align*} \frac{f\left(\mathbf{x}\mid\alpha=2\beta,\beta\right)}{f\left(\mathbf{y}\mid\alpha=2\beta,\beta\right)} &=\frac{\exp\left[\left(\beta-1\right)\sum_{i=1}^n\log\left(x_i^2(1-x_i)\right)+n\log\left(\frac{\Gamma\left(3\beta\right)}{\Gamma\left(2\beta\right)\left(\beta\right)}\right)\right]\exp\left(\sum_{i=1}^n\log(x_i)\right)}{\exp\left[\left(\beta-1\right)\sum_{i=1}^n\log\left(y_i^2(1-y_i)\right)+n\log\left(\frac{\Gamma\left(3\beta\right)}{\Gamma\left(2\beta\right)\left(\beta\right)}\right)\right]\exp\left(\sum_{i=1}^n\log(y_i)\right)}\\\\ &=\exp\left[\left(\beta-1\right)\left[\sum_{i=1}^n\log\left(x_i^2(1-x_i)\right)-\sum_{i=1}^n\log\left(y_i^2(1-y_i)\right)\right]+\left[\sum_{i=1}^n\log(x_i)-\sum_{i=1}^n\log(y_i)\right]\right] \end{align*}$$
does not depend on $\beta$ if and only if $T(\mathbf{x})=\sum_{i=1}^n\log\left(x_i^2(1-x_i)\right)$ equals $T(\mathbf{y})=\sum_{i=1}^n\log\left(y_i^2(1-y_i)\right)$
$c)$ We have that $T(\mathbf{X})=\left(\sum_{i=1}^n\log(X_i),\sum_{i=1}^n\log(1-X_i)\right)$ is sufficient for the family $\{B(\alpha,\beta),\alpha=\beta^2,\beta>0\}$ since for all $\mathbf{x}\in\mathbb R^n$ and $\beta>0$,
$$\begin{align*} f(\mathbf{x}\mid\alpha=\beta^2,\beta) &=\left[\frac{\Gamma\left(\beta^2+\beta\right)}{\Gamma\left(\beta^2\right)\left(\beta\right)}\right]^n\left(\prod_{i=1}^n x_i\right)^{\beta^2-1}\left(\prod_{i=1}^n (1-x_i)\right)^{\beta-1}\prod_{i=1}^n\mathbb{1}_{[0,1]}(x_i)\\\\ &=\underbrace{\exp\left[\left(\beta^2-1\right)\sum_{i=1}^n\log(x_i)+(\beta-1)\sum_{i=1}^n\log(1-x_i)+n\log\left(\frac{\Gamma\left(\beta^2+\beta\right)}{\Gamma\left(\beta^2\right)\left(\beta\right)}\right)\right]}_{g(T(\mathbf{x})\mid\alpha=\beta^2,\beta)}\underbrace{\mathbb{1}_{[0,1]}(x_{(1)})\mathbb{1}_{[0,1]}(x_{(n)})}_{h(\mathbf{x})} \end{align*}$$
We also have that $T(\mathbf{X})=\left(\sum_{i=1}^n\log(X_i),\sum_{i=1}^n\log(1-X_i)\right)$ is a minimal sufficient statistic since for any two possible sample points $\mathbf{x}$ and $\mathbf{y}$
$$\begin{align*} \frac{f\left(\mathbf{x}\mid\alpha=\beta^2,\beta\right)}{f\left(\mathbf{y}\mid\alpha=\beta^2,\beta\right)} &=\frac{\exp\left[\left(\beta^2-1\right)\sum_{i=1}^n\log(x_i)+(\beta-1)\sum_{i=1}^n\log(1-x_i)+n\log\left(\frac{\Gamma\left(\beta^2+\beta\right)}{\Gamma\left(\beta^2\right)\left(\beta\right)}\right)\right]}{\exp\left[\left(\beta^2-1\right)\sum_{i=1}^n\log(y_i)+(\beta-1)\sum_{i=1}^n\log(1-y_i)+n\log\left(\frac{\Gamma\left(\beta^2+\beta\right)}{\Gamma\left(\beta^2\right)\left(\beta\right)}\right)\right]}\\\\ &=\exp\left[\left(\beta^2-1\right)\left[\sum_{i=1}^n\log(x_i)-\sum_{i=1}^n\log(y_i)\right]+(\beta-1)\left[\sum_{i=1}^n\log(1-x_i)-\sum_{i=1}^n\log(1-y_i)\right]\right] \end{align*}$$
does not depend on $\beta$ if and only if $T(\mathbf{x})=\left(\sum_{i=1}^n\log(x_i),\sum_{i=1}^n\log(1-x_i)\right)$ equals $T(\mathbf{y})=\left(\sum_{i=1}^n\log(y_i),\sum_{i=1}^n\log(1-y_i)\right)$