In Algorithms for Non-negative Matrix Factorization, Lee and Seung give multiplicative algorithms derived from gradient descent on the Frobenius norm to find a non-negative matrix factorization.
Elsewhere, I've seen iterative algorithms where the convergence is proved not via gradient descent but via some contraction mapping fixed point theorem or something like that. I'm wondering whether every iterative algorithm can be rewritten as gradient descent on some objective function.