Long time scale correlation

Question

I have some EXTREMELY noisy data (standard deviation a is greater than the mean), but plotting it with a 15 data point running average does well to get a visual indication of the trending. I want to see if any of my data sets are correlated, but standard correlation gives me very poor results due to the noisiness of the data (and the fact that I care about trends over scales on the order of hundreds of data points). I'm thinking of maybe just cranking the running average up to a few hundred data points and then running the correlation on those, but something makes me think that there might be a better way.

Answer 1

Your use of a "running average" sounds like you have some kind of time series data. If so may want to look at cointegration rather than correlation to analyze it. You can find a good discussion of the differences between correlation and cointegration on SE's Quantitative Finance Beta site. See:How are correlation and cointegration related?

Not original to me, but the simple comparative description in the mentioned post that best captures the difference for me came from the contributor, isomorphismes:

• Correlation is a property of collections of observations.
• Cointegration is a property of time series.

The important difference is that temporal observations have one neighbour to their left and one to their right. Collections are like a set — no implicit "neighbour" relationships.

Moving average is an inappropriate statistic to apply to lab experiments or phone survey data. It is appropriate in the analysis of time series.

and a comment:

Interesting point to make. If we are comparing two time series, correlation tell us something about the complete time series as a whole, whereas cointegration tells us something about the individual matching points.

from user, Gravitas.

If you still want to stay with correlation consider applying it to the log of the differences at each time step.