Gradient Descent for Exponential Functions

Question

I am trying to develop a non-linear regression for several functions (power, log and exponential). the idea was to use a log transformation to get an initial set of points, close enough to the real values in order to help the gradient descent (there are multiple minima).

The question is, this approach does not give me a valid result. I share my code here, maybe is not the correct way to do it, but it is a start.

public static ER er(final double x[],
final double y[], final int bX, final int bY, final int l) {
  double sx = 0.0, sy = 0.0, xy = 0.0, x2 = 0.0;

  for (int i = 0; i < l; i++) {
    double v = Math.log(y[i + bY]);
    sx += x[i + bX];
    sy += v;
    xy += x[i + bX] * v;
    x2 += Math.pow(x[i + bX], 2);
  }

  double d = l * x2 - Math.pow(sx, 2.0),
    m = (l * xy - sx * sy) / d,
    b = (sy * x2 - sx * xy) / d,
    my = sy / l, sse = 0.0, sst = 0.0;

  for (int i = 0; i < l; i++) {
    double v = Math.log(y[i + bY]), f = m * x[i + bX] + b;
    sse += Math.pow(v - f, 2);
    sst += Math.pow(v - my, 2);
  }

  return new ER(Math.exp(b), m, 1 - (sse / sst), Math.sqrt(1.0/(l-1.0)*sse));
}


public static class ER extends BRM{
  protected final double a, b, r2, s;
  public ER(final double a, final double b, final double r2,
  final double s) {
    this.a = a;
    this.b = b;
    this.r2 = r2;
    this.s = s;
  }

  public double solve(final double x) {
    return a * Math.exp(x * b);
  }

  @Override
  public String toString() {
    return "f(x) = "+a+"e^"+b+"x ("+r2+", "+s+")";
  }
}

public static ER er2(final double x[], final double y[]) {
  System.out.println("X = "+ PrintUtils.array(x));
  System.out.println("Y = "+ PrintUtils.array(y));
  ER theta = er(x,y);
  System.out.println("Log-Linear Theta "+theta);
  double a = theta.a, b = theta.b;

  for (int c = 0; c < 1; c++) {
    double gradient_a = 0.0, gradient_b = 0.0;
    for (int i = 0; i < x.length; i++) {
      double delta = a * Math.exp(x[i] * b);
      double residual = y[i] - delta;
      System.out.println("("+y[i]+", "+delta+", "+residual+")");
      gradient_a += -2.0 * residual * Math.exp(b*x[i]);
      gradient_b += -2.0 * residual * a * x[i] * Math.exp(b*x[i]);
    }
    gradient_a /= x.length;
    gradient_b /= x.length;

    System.out.println("Gradient ("+gradient_a+", "+gradient_b+")");
    a -= gradient_a * 0.0001;
    b -= gradient_b * 0.0001;
  }
  System.out.println("Non-linear theta "+new ER(a, b, theta.r2, theta.s));
  return new ER(a, b, theta.r2, theta.s);
}

---

I will try to express my problem with equations. I want to find the best possible fit using an exponential function: $f(x, a, b)=a \times e^{b \times x}$ As such I want to minimize the following expression: $$S = \sum_{i=0}^{m} r_i^2$$ With the resdiual equal to: $r_i=y_i - f(x_i, a, b)$

My idea was a rather simple one and probably done already, is to use a log transformation to find an initial set of values. These values close to the correct solution. And then run a Gradient Descent, as a tunning algorithm just to improve the initial guess.

I will write my gradient and update rules:

$$gradient_{a_{i}} = -2 \times r_i \times e^{b\times x_i}$$ $$gradient_{b_{i}} = -2 \times r_i \times a \times x_i \times e^{b\times x_i}$$

$$gradient_a = \frac{\sum_{i=0}^{m} gradient_{a_{i}}}{m}$$ $$gradient_b = \frac{\sum_{i=0}^{m} gradient_{b_{i}}}{m}$$

And finally, the update rules are the following:

$$a_{t+1} = a_t - gradient_a \times \delta$$ $$b_{t+1} = b_t - gradient_b \times \delta$$

Is there an error in my rationale?