Minimize function with respect to constraint

61 Views Asked by At

Consider the function $ f(a) =\sum_{k=1}^N \|a- x_k\|^2$, where $a, x_k \in \mathbb{R}^d$, which we want to minimize w.r.t. $\|a- c\|^2=1$

Building the lagrangian yields: $\sum_{k=1}^N \|a- x_k\|^2 + \lambda (1-\|a- c\|^2) $ Taking derivative w.r.t. $a$ yields: $$ N a - \sum_{k=1}^N x_k - \lambda (a- c) =0 $$ U Dividing by $N$ yields: $$ (a - m) - \lambda' (a-c)=0 $$, where $m:= 1/N\sum_{k=1}^N x_k$ and $\lambda' = \lambda/N$ Using hint in comments: $$a (1- \lambda') +\lambda'c - m=0 $$ $$ a = \frac{m-\lambda'c}{1-\lambda'} (*)$$ Using the constraint, we get:

$$\|\frac{m-c}{1- \lambda'}\|^2 =1 \Rightarrow 1 - \pm \| m-c\| = \lambda'$$

Putting that in (*) yields: $\frac{m - (1 - \pm \| m-c\|)c }{\mp \| m-c\|}$

How do I find the solution then?

1

There are 1 best solutions below

0
On

Another way to solve this problem is to write $$ \mathbf{a}=\mathbf{c}+\frac{1}{\| \mathbf{u} \|} \mathbf{u}$$ The objective function now reads $$ \phi(\mathbf{u}) = \sum_n \| \mathbf{b}_n + \frac{1}{\| \mathbf{u} \|} \mathbf{u} \|^2 $$ where $\mathbf{b}_n=\mathbf{c}-\mathbf{x}_n$.

The gradient is (for one element) $$ \frac{2}{\| \mathbf{u} \|} \left[ \mathbf{b}_n - \frac{\mathbf{b}_n^T \mathbf{u}}{\| \mathbf{u} \|^2} \mathbf{u} \right] $$

Finally the solution is a multiple of $\sum_n \mathbf{b}_n = N (\mathbf{c} - \bar{\mathbf{x}} )$ and thus $$ \mathbf{a}=\mathbf{c}+\frac{1}{\| \mathbf{c} - \bar{\mathbf{x}} \|} (\mathbf{c} - \bar{\mathbf{x}} )$$