optimization problem with two lagrange multipliers (proof that OLS is BLUE)

159 Views Asked by At

enter image description here

I have some difficulty in understanding the intermediate step to get $a_i$.

Attempt: we have the following objective function: $$\sum_{i=1}^n a_i^2 -\lambda \sum_{i=1}^n a_i - \mu\sum_{i=1}^n a_ix_i, $$ where $\lambda$ and $\mu$ are lagrange multipliers. Differentiating with respect to $a_i, \lambda$, and $\mu$ yield that $2a_i -\lambda -\mu x_i$ and $\sum_{i=1}^n a_i$ and $\sum_{i=1}^n a_ix_i$, respectively. If we let these equations equal to zero, the second and third equations are just two constraints. I think I should use the first equation to find $a_i$, but I am stuck here. Can anyone give me some help?

1

There are 1 best solutions below

3
On

You have $$\label{eqn1} 2a_i=\lambda+\mu x_i\tag{1} $$ Imposing the constraints, $$ n\lambda+\mu \sum x_i = 0,\quad\lambda\sum x_i+\mu\sum x_i^2=2 \sum a_ix_i=2 $$ we get $$ \lambda=\frac{2\sum x_i}{(\sum x_i)^2-n\sum x_i^2},\quad \mu=\frac{2n}{n\sum x_i^2-(\sum x_i)^2}. $$ Substituting back into \eqref{eqn1} $$ a_i=\frac{-\sum x_i+n x_i}{n\sum x_i^2-(\sum x_i)^2}=\frac{x_i-\bar{x}}{\sum (x_i-\bar{x})^2}. $$