I am trying to figure out how to implement a cyclic running Regression Line computing algorithm using Least Squares for streaming time series data in the most efficient way. In other words, having $LS(s)$ im looking for a way to compute $LS(s)$ after adding an element to s and removing the kth oldest element from it.
Given a set $s$ of $x,y$ samples I was thinking maybe it is possible to compute $LS(s\bigcup\{a\})$ (for some new sample $a$) and $LS(s-\{a\})$ (for some sample $a$ in $s$, specifically the oldest sample) given $LS(s)$, is there a known solution to this?
Assuming that I properly understand.
If you use normal equations for the regression $y=a+bx$, $a$ and $b$ are given solving the equations $$S_y=n a+b S_x$$ $$S_{xy}=a S_x+b S_{xx}$$ Now, you want to add point $n+1$ and remove point $1$ from the regression set. So, the equations are now $$S'_y=n a'+b' S'_x$$ $$S'_{xy}=a' S'_x+b' S'_{xx}$$Computing the terms we have $$S'_y=S_y+y_{n+1}-y_1$$ $$S'_x=S_x+x_{n+1}-x_1$$ $$S'_{xy}=S_{xy}+x_{n+1}y_{n+1}-x_1y_1$$ $$S'_{xx}=S_{xx}+x_{n+1}^2-x_1^2$$ which make the updating process quite simple.
For sure, when this is done, do not forget to replace make the update $S=S'$ and to shift the indices $k=k-1$ to be ready for the next step.