MLE for a regression with alpha = 0

219 Views Asked by At

Consider the model $Y_i = \beta x_i + \varepsilon_i$ where $i = 1,\ldots, n$.

We know that $\varepsilon_1,\ldots, \varepsilon_n$ is iid sequence of random variables from $N(0,\sigma^2)$ and $x_i, i = 1,\ldots,n$ are given constants. Find MLE for both $\beta$ and $\sigma^2$. Find the distribution of $\hat \beta$. Is $\hat \beta$ an unbiased estimator?

So, basically, this is what I've done so far:

$$L(\beta,\sigma^2) = \left(\frac{1}{2\pi\sigma^2}\right)^{\frac{n}{2}} \exp\left[\frac{\sum(Y_i - \beta x_i)^2}{2\sigma^2} \right] \\ \frac{d(\ln L)}{d\beta} = 0 = \sum(Y_i - \beta x_i)(x_i) \implies \hat \beta = \sum \frac{Y_i}{x_i}$$

I'm not sure if I did this correctly, but if someone could find the errors, that would be helpful.

I also tried to find the MLE for $\sigma^2$:

$$\hat \sigma^2 = \frac {\sum (Y_i - \beta x_i)^2}{n}$$

I'm not quite sure if I'm doing any of these questions correctly, and I am not really sure how to figure out the distribution for $\hat \beta$ and whether it is an unbiased estimator or not. Like how can I determine the distribution of $\beta$ with the given information? Thanks!

Edit (To show more information):

$\sum x_i y_i - \beta \sum x_i^2 = 0 \implies \beta \sum x_i^2 = \sum x_i y_i \implies \beta = \frac{\sum x_i y_i}{\sum x_i^2} \implies \beta = \frac{y_i}{x_i}$.

2

There are 2 best solutions below

2
On

Dont have enough to comment, but you'll get the right answers if you are bit more careful. i.e. it should be $\Sigma (Y_i - \beta x_i)x_i$ that is, $x_i$ is within then sum so doesnt cancel out.

0
On

$$ \widehat\beta = \frac{\sum_{i=1}^n x_iY_i}{\sum_{i=1}^n x_i^2}. $$ The $Y$ in the expression above is capital; the $x$s are in lower case. That is to emphasize that $Y_i,$ in the problem as stated, is a random variable, and $x_i$ is not random. Therefore we have \begin{align} & \operatorname E\widehat\beta = \frac{\sum_{i=1}^n x_i \operatorname EY_i}{\sum_{i=1}^n x_i^2} = \frac{\sum_{i=1}^nx_i(\beta x_i)}{\sum_{i=1}^nx_i^2} = \beta. \\[12pt] & \operatorname{var}\widehat\beta = \frac{\sum_{i=1}^n x_i^2\operatorname{var}Y_i}{\left( \sum_{i=1}^n x_i^2 \right)^2} = \frac{\sum_{i=1}^n x_i^2 \sigma^2}{\left( \sum_{i=1}^n x_i^2\right)^2} = \frac{\sigma^2}{\sum_{i=1}^n x_i^2}. \end{align}

The only thing you need to know beyond that is that a linear combination of normally distributed random variables with constant (i.e. non-random) coefficients is normally distributed.