Consider a simple M/M/1 queue (for example), with arrival rate $\lambda$ and service rate $\mu$. Under the stability condition ($\lambda<\mu$), Queueing Theory says that the average queue length in long run is $$E(L)=\frac{\rho^2}{1-\rho},$$ where $\rho=\frac{\lambda}{\mu}$. This is a common result.
However, to my surprise, I could not find any theory or papers (or maybe I could not catch the keyword to look for) that tackle the following seemingly simple question: What is the average queue length after the system has been run for the first $t$ time units (with the stability condition relaxed).
I would really appreciate any references.