Which solver method for solving indicator function in a QP problem

75 Views Asked by At

I am trying to solve below QP problem with Heaviside step function: determining z is to make the J(z) function achieve a maximum

$$ \begin{align} \min_z J(z)=(\sum_{(x_i,y_i)\in T,y_i=1} I(y_if(x_i,z))) \times (\sum_{(x_i,y_i)\in T,y_i=-1} I(y_if(x_i,z)))\\ \text{where: Heaviside step function: } I(u)=\begin{cases} 1, & \text{if $u \ge 0$} \\ 0, & \text{if $u \lt 0$} \end{cases} \\ \text{and:} \\ f(x,z)= \sum_{x_p \in SV^+,y_p=1} z_p\alpha_py_p(x.x_p)+\sum_{x_n \in SV^-,y_n=-1} \alpha_ny_n(x.x_n) +b, \\ SV^+, SV^-,\alpha_n,\alpha_p,x_n,x_p \text{ is constant value}, y_p=1,y_n=-1\\ \end{align} $$

I have an idea to cross this problem by approximate I to Logistic function, e.g: $$ \begin{align} I(u)=\frac{1 }{1+e^{-2ku}} \text{where a larger k corresponds to a sharper transition at u = 0} \end{align} $$ Then I can find z by using Gradient Descent method:

$$ \begin{align} z_i=z_i^{old} + \eta * \nabla J(z_i^{old}) \\ \text{where } \eta \text{ is learning rate} \\ \end{align} $$

Could anyone give me a some advices about my above analysis for search z optimal from J(z)?

Thank everyone