The specific problem I'm trying to solve is: $$h_k(x, n) = \left(\frac{\alpha}{n} + 1 - \alpha\right) \sum_{i=0}^{k} c_ix^i.$$
Given $k$ and a stream of tuples $(x, n, h_k(x, n))$ (where the $x$'s and $n$'s are not necessarily distinct), find an $\alpha$ and $c_0,c_1,\ldots,c_k$ that approximate $h_k$. The simplest solutions to fitting such a curve would be to use something like least squares, but that would require storing all tuples and computing the approximation at the end.
Is there a way to keep a rolling approximation of the coefficients and $\alpha$, so that the approximation can be updated as new tuples come in, without needing to store all previous tuples?
I came across this question while researching something similar. Perhaps this will be of use : http://erikerlandson.github.io/blog/2012/07/05/deriving-an-incremental-form-of-the-polynomial-regression-equations/
The regression is $$\left( \begin{array} {c} a_0 \\ a_1 \\ \vdots \\ a_m \\ \end{array} \right) = \left( \begin{array} {cccc} n & \Sigma x & \cdots & \Sigma x^m \\ \Sigma x & \Sigma x^2 & \cdots & \Sigma x^{m+1} \\ \vdots & & \ddots & \vdots \\ \Sigma x^m & \Sigma x^{m+1} & \cdots & \Sigma x^{2 m} \\ \end{array} \right) ^ {-1} \left( \begin{array} {c} \Sigma y \\ \Sigma x y \\ \vdots \\ \Sigma x^m y \\ \end{array} \right)$$
so you need to store $\Theta(k^2)$ values for incremental calculation.