Gradient descent for periodic function

Question

Problem:

minimize E=$\sum_{t=0}^T [ Y(t)-\sum_{k=0}^K(X(t+k)*cos(k*F+PHI)) ]^2$

where F, PHI - has to be optimized;

Y(t), X(t) 1D arrays are given;

t=0...T; T=1000;

k=0..K; K=10;

I can use FFT and get direct estimation of F and PHi (and it works fine), but my question is: are there other methods which can work in time domain using iterative approach? I tried to use mini-batch gradient descent, but because the problem has a lot of local minimums, it rarely converges (and strongly depend on initialization).

Answer 1

Edit: My main point is that

it's possible to rewrite this as a sum $\sum\limits_{i}g_{i}(F,\phi)$. (with possibly a very large number of terms) where, each of these functions has a Lipschitz continuous derivative. This essentially comes from the fact that $\cos$ and it's derivatives are bounded, and the fact that the parameters only interact through linear combinations. That means you can apply gradient descent with a constant step size.

For details, see p.144 at http://www.stat.purdue.edu/~vishy/introml/notes/Optimization.pdf

In any case, that is just show it's possible to design an algorithm which is guaranteed to reach some local minimum, but it depends on where you start.