I was reading the paragraph in section E.4 of this paper, where the authors discuss adding longer histories in neural/numerical PDE solvers and claim that "using higher-order differences [in time] within an explicit autoregressive scheme is known to deteriorate the rollout stability and introduce exponentially increasing errors over time."
What are the reasons for this reduced stability? I can think of one possible reason: as we include higher-order differences, the condition for stability becomes more restrictive, and we require smaller stepsizes in time. Are there any other reasons?