Stationary accumulation points

Question

I have been reading about trying to prove global convergence of general optimization alrgoithms and am have come across the term "stationary accumulation point" and am trying to decipher exactly what that means. I read the definition of an accumulation point on Wikipedia (also called a limit point I guess) but am having trouble with what it means in an optimization context.

Answer 1

In the context of optimization, you typically have a function $f$ that you wish to maximise/minimise. A stationary point of $f$ is any point at which the gradient/derivative of $f$ is zero. In practice, to find such a stationary point, you would typically use an iterative algorithm which generates a sequence of points $x_n$ for $n=1,2,...$. For such algorithms we would hope (and maybe we can prove) that $x_n$ converges to a stationary point as $n\to\infty$, or at least we would hope that the desired stationary point is an accumulation point of the sequence $x_n$; this just means that a subsequence of the $x_n$ converges to the stationary point (which allows for situations where the sequence might oscillate between two or more points). So a stationary accumulation point is a stationary point of $f$ which is also an accumulation point ('cluster point') of the sequence $x_n$. In practice, we can't actually compute the whole sequence, so we just compute $x_N$ for large $N$; hopefully the gradient at $x_N$ will be approximately zero.

See also this question.