Characterize the minimizers x*

64 Views Asked by At

While solving for least square case,

$\sum_{i=1}^m$$($$\sum_{j=1}^n$ $A_{i,j}$$x_{j}$ $-$ $y_{i}$)$^2$

the minimum value of $x$ in $||Ax-y||^2$ is characterized by the normal equation $A^TAx^*$ $=$ $A^Ty$,

Now what will be the impact if suppose if we add non-negative and non-zero coefficients to this expression?

$\sum_{i=1}^m w_i$$($$\sum_{j=1}^n$ $A_{i,j}$$x_{j}$ $-$ $y_{i}$)$^2$

How one could characterize the minimum of $x^*$ which satisfies the above expression.?

I would find like to know the impact of the weighted w's in the least square expression. I found like the minimizer x* is $A^TwAx^*$ $=$ $A^Twy$ .I wanted to confirm this case in particular

1

There are 1 best solutions below

4
On BEST ANSWER

Let $\tilde{A}_{ij}=A_{ij}w_i^{1/2}$ and $\tilde{y}_i=w_i^{1/2}y_i.$ Then the expression takes the form $$\sum_{i=1}^m\left (\sum_{j=1}^n\tilde{A}_{ij}x_j-\tilde{y}_i\right )^2$$ Let $Dy=\{w_i^{1/2}y_i\}_{i=1}^m.$ Then $\tilde{y}=Dy.$ Moreover $\tilde{A}=DA.$ Hence the condition $\tilde{A}^T\tilde{A}x=\tilde{A}^T\tilde{y}$ takes the form $$A^TD^2Ax=A^TD^2y$$ So the result from your first comment is correct.