Unsupervised learning algorithms to detect anomaly in waves.

179 Views Asked by At

I have a sample of graphs (more than 10000...). that look like in the image below: enter image description here

I am searching for an unsupervised learning algorithms that can help me to detect anomalous observations.

Here what I suggest for beginning: for every observation I have a collection of points $(x,y)$. With this collection, I find Fourier series with regression (I compute coefficients with the base $\{1,\sin(x),\cos(x),\sin(2x),\cos(2x)\dots\}$). Now I have a set of coefficients instead of waves.

Somebody have an idea how to detect anomaly?

1

There are 1 best solutions below

1
On

FFT (Fast Fourier Transform) each wave to get its Power spectral density (PSD).
Your "anomalous" waves may transform to a PSD like that of white noise, flat (however I'm no expert).

To refine this, make a Linear classifier. Train it manually: pick out anomalous 10 ? out of 100 ? waves by hand and eye, and make a linear classifier with e.g. scikit-learn to separate the 10 from the 90 / 10k from 90k.
Added 1 Dec: Spectral flatness is a measure of how noise-like a signal is; it's easy to implement, and may be adequate for your task.

(I'd suggest asking over on dsp.stackexchange.com, with tag Matlab or Python too.
Moderators, move the question ?)