What is time-frequency plane and when do we use it?

Question

I have seen frequency-amplitude plane for fourier analysis however this time-frequency plane makes no sense to be. It lacks the amplitude information. What is the way to understand and interpret this a time-frequency plane and when/why do we need it at all?

Answer 1

The time-frequency plane comes up in a few different forms, but the simplest way to understand it is via short-time (or windowed) Fourier transforms. The STFT is defined to be

$$\mathcal{V}_gf(\omega, t) = \int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi}} e^{-i\omega t} g(t-\tau)f(\tau)\,d\tau$$

where $f,g$ are suitably nice functions (e.g. $f,g\in L^1(\mathbb{R})$ and $fg_t\in L^1(\mathbb{R})$). $g$ is called the windowing function, often taken to be a Gaussian, and $f$ is the signal you're analyzing. If $g \equiv 1$ this would of course just be the usual Fourier transform.

The role of $g$ here is to "cut out" portions of $f$ to analyze. The STFT's goal is to give you not only frequency information but temporal information. The usual Fourier transform can tell you that a certain range of frequencies is present and how prevalent they are (the relative area), but it cannot tell you at what time in the signal those frequencies occurred. The STFT's goal is to remedy this: you get local frequency information. Meaning that you can say with some certainty that certain frequencies occurred in a certain time period.

Two often used examples of STFTs are sheet music and equalizers. Sheet music tells you what notes to play at what times, meaning you have both temporal and frequency information. Equalizers show you what range of frequencies are occurring at the current time.

The time-frequency plane is then just the $(\omega,t)$ space that the STFT is built upon. Of course there are other ways to get to a time-frequency plane like doing a temporally-varying wavelet decomposition or something like that. The amplitude information is still there though, but now it is just in the value of $\mathcal{V}_gf$. It in some sense is like a three-dimensional space where the $xy$ plane is $(\omega,t)$ and the $z$ component is $|\mathcal{V}_gf|$.

As for why we need it, when you're doing a naive frequency analysis, you can often make some critical mistakes. Suppose that you're an audio engineer and you want to touch up a recorded performance by the band you're working with. There will be times where the singer accidentally hits a wrong note very briefly or there's an accidental cymbal crash that shouldn't be there. Well you've recorded a nearly perfect performance outside of a few minor details. If you did a naive filtering by just taking a Fourier transform and filtering out the frequency that you didn't want, you will remove that frequency from the whole song. That's not what you want at all. You want to be able to take that cymbal crash out at that specific moment. To accomplish this, you need to keep the temporal data. The STFT is what does this for you. There are also some nice uses for the STFT in geophysics when it comes to first arrivals of primary and secondary waves and such.