Let $N$ be the number of data points. Let $\boldsymbol{\Sigma}_0$ and $\boldsymbol{\Sigma}_X$ be fixed covariance matrices. I have an online algorithm for discrete time that computes
$$ \boldsymbol{\Sigma}_N = (\boldsymbol{\Sigma}_0 ^{-1} + N \boldsymbol{\Sigma}_X^{-1})^{-1} $$
at each time step. Thus, $N$ increases by $1$ at the rate of the data stream. I would prefer to not compute a matrix inversion at each time step, since the dimension of the $\boldsymbol{\Sigma}$'s may be large. Is it possible to do an efficient update without using a matrix inversion?
$$ \begin{aligned} \boldsymbol{\Sigma}_N &= (\boldsymbol{\Sigma}_0 ^{-1} + N \boldsymbol{\Sigma}_X^{-1})^{-1} \\ &\Downarrow \\ \boldsymbol{\Sigma}_{N+1} &= (\boldsymbol{\Sigma}_0 ^{-1} + (N+1) \boldsymbol{\Sigma}_X^{-1})^{-1}. \end{aligned} $$
It is easy to imagine fast updates to the precision matrices,
$$ \begin{aligned} \boldsymbol{\Lambda}_N &= \boldsymbol{\Sigma}_0 ^{-1} + N \boldsymbol{\Sigma}_X^{-1} \\ &\Downarrow \\ \boldsymbol{\Lambda}_{N+1} &= \boldsymbol{\Lambda}_N - N \boldsymbol{\Sigma}_X^{-1} + (N+1) \boldsymbol{\Sigma}_X^{-1}. \end{aligned} $$
But I'm stuck inverting this if I need $\boldsymbol{\Sigma}_{N+1}$.