Solving the euclidian distance squared to kernelize a Lagrangian dual

48 Views Asked by At

Homework question, looking for a hint on the following problem: I'm trying to solve this dual lagranging form (which could potentially be wrong already, but let's assume it is right)

$\boldsymbol{x}$ is a set of datapoints in R^2, $\{\alpha_i\}$ are lagrange multipliers. \begin{align*} \mathcal{L} = \sum^N_{i=1}\alpha_i\left\|\boldsymbol{x}_i - \sum^N_{j=1}\alpha_j\boldsymbol{x}_j\right\|^2 \end{align*} By substituting $\sum^N_{j=1}\alpha_j\boldsymbol{x}_j$ with $\mathcal{S}$, I then get \begin{align*} \mathcal{L} &= \sum^N_{i=1}\alpha_i\left\|\boldsymbol{x}_i - \mathcal{S}\right\|^2 \\ &= \sum^N_{i=1}\alpha_i\left((\boldsymbol{x}_{i,1} - \mathcal{S})^2 + (\boldsymbol{x}_{i,2} - \mathcal{S})^2 \right) \\ &= \sum^N_{i=1}\alpha_i\left( \boldsymbol{x}_{i,1}^2 -2\mathcal{S}\boldsymbol{x}_{i,1} + \mathcal{S}^2 \boldsymbol{x}_{i,2}^2 -2\mathcal{S}\boldsymbol{x}_{i,2} \mathcal{S}^2 \right) \\ &= \sum^N_{i=1}\alpha_i\left( \boldsymbol{x}_{i,1}^2 + \boldsymbol{x}_{i,2}^2 -2\mathcal{S}(\boldsymbol{x}_{i,1} + \boldsymbol{x}_{i,2}) + 2\mathcal{S}^2 \right)\\ &= \sum^N_{i=1}\alpha_i\left( \|\boldsymbol{x}_i\|^2 -2(\sum^N_{j=1}\alpha_j\boldsymbol{x}_j)(\boldsymbol{x}_{i,1} + \boldsymbol{x}_{i,2}) + 2(\sum^N_{j=1}\alpha_j\boldsymbol{x}_j)^2 \right) \end{align*}

I want to substitute part of this with a kernel $k(x_n,x_m)$, but for this I need some kind of double sum over the datapoints. Is there a step I can take to get the formula in this form, or am I on the wrong track and should I work out the L2 norm in a different way? It's a homework question, so I would appreciate just a nudge in the right direction.

1

There are 1 best solutions below

0
On

Got a hint to solve $\sum^N_{i=1} \alpha_i(\boldsymbol{x}_i - \boldsymbol{S})^T(\boldsymbol{x}_i - \boldsymbol{S})$ instead.