Problem: Obtain the autocorrelation of unevenly(!) sampled time series.
Data (Time series) description: ~$10^6$ unevenly sampled time series, each containing ~$10^4$ data points at irregular intervals.
Distribution of the interval (gap) length:
- 12 seconds : most common (minimum)
- 12-18 hours : very common
- 1-5 days : occasionally
- 1 week - 3 months : rare
The data is periodic (but the amplitudes are not constant) with periods from 1 hour - 100 days and thus on the same order as many of the gaps. The shape of the signals is not always sinusoidal.
Standard approaches (that do not work):
Lamb-Scargle Fourier Transform: This method does not work since the signals are not sinusoidal, but can contain sharp peaks etc.
(Linear) interpolation (then use standard time series analysis): This method does not work because the gaps are too large and too common, since whole oscillations are missing.
Gaussian process regression/ Kernel methods: They can be made to work - however they are unfortunately much too slow to compute the autocorrelation of all ~$10^6$ time series.
EDIT: I am at this moment unable to publish any sample data sets publicly.