Can I Fourier-transform a list of separate events?

271 Views Asked by At

As a non-mathematican, this is way over my head and I miss the terminology, so please bear with me:

An IT system we operate throws discrete events in seemingly random time intervals. We try to analyze the reasons. If I counted these events per 5-minute-block, I would get something like 1,1,3,1,2,7,2,1,5... If I counted them per 30-second-block, it would be more like 0,0,0,1,0,0,3,0,0,1,0,...

Assuming that there are multiple sources which would generate these events on their own (yet to be seen), we should be able to do a frequency analysis? What I ultimately want is something like "There are 3 sources of these events. One repeating every 39 seconds, one repeating every 73 seconds, and one repeating every 117 seconds." From there on, I could track down the processes with this repetition pattern.

From what I understand after 3blue1brown, with Fourier-Transformation, this might be possible... But I have no idea where to even start. Please point me into the right direction.

1

There are 1 best solutions below

0
On BEST ANSWER

Fast Fourier transform (FFT) computes for the sinusoid components of your signal, but what you need is periodicity of your signal "in general". I think you need to compute the autocorrelation of your signal instead, then find the peaks.

Here is a sample (lazy) Python code:

import numpy as np
import matplotlib.pyplot as plt
from scipy.signal import find_peaks

sig = np.sin(np.arange(50))   # sample signal: sine
autocor = np.correlate(sig, sig, mode="full")   # autocorrelation is a EVEN function
autocor = autocor[autocor.size // 2:]   # so let's remove the negative mirror image
plt.plot(autocor, '-o')   # plot autocorrelation

peaks, _ = find_peaks(autocor)
print(peaks)

The peaks it gave are $[6,13,19,25,31,38,44]$. This means that the input seems to be periodic every $6, 13, 19, \ldots$ units. For sine wave, this makes sense because np.sin interprets input in radians, and sine is periodic every $2\pi \approx 6.28$ radians. The other values seem to be approximate multiples of $2\pi$, so in this example they are consequence of $2\pi$ periodicity. However, you should be more careful in your case because we do not know the pattern.

Side Note: The output given by np.correlate function is not normalized, and this is the one I encountered in my studies. I guess normalizing it may provide more insights, but I don't know... Anyways, here is the code snippet from above with additional lines for normalization.

sig = np.sin(np.arange(50))   # sample signal: sine
sig -= np.mean(sig)   # <--- Normalization
autocor = np.correlate(sig, sig, mode="full")   # autocorrelation is a EVEN function
autocor = autocor[autocor.size // 2:]   # so let's remove the negative mirror image
autocor /= autocor[0]   # <--- Normalization (note: autocor[0] is the variance)
plt.plot(autocor, '-o')   # plot autocorrelation