How to find the variance of the least squares method?

935 Views Asked by At

I'm using Bayesian Inference to find the parameters for a specific formula $ Y=a*X^b$.

Then; $Y$ ~ $N(a*X^b, variance)$

First I want to use least squares method to find the parameters $a$ en $b$.Then I want to find the variance that belong to these parameters, like the formula above.

Does anyone have an idea how to find that variance?

I know the variance should be $ \frac {\sum(x_i - mean)^2} N$

But what should be $x_i$ and what is the mean?

1

There are 1 best solutions below

2
On

Taking logarithms on both sides, we get \begin{equation} \log Y_i = \log a X_i^b = \log a + b \log X_i \end{equation} Calling \begin{align} y_i &= \log Y_i \\ x_i &= \log X_i \\ \beta_0 &= \log a \\ \beta_1 &= b \end{align} You'd get a linear model \begin{equation} y_i = \beta_0 + \beta_1 x_i \end{equation} in which you could apply simple linear regression to estimate $\beta_0,\beta_1$,denoted as $\hat{\beta}_0 , \hat{\beta}_1$. Your variance could be estimated as $$variance =\sum_{i=1}^n \frac{[y_i - (\hat{\beta}_0 + \hat{\beta}_1 x_i)]^2}{n-2}$$ or you could divide by $n$ for large $n$.