preliminary evaluation for forecasting models

Question

Suppose I would like to use a method for data prediction, and that I have some empirical data (i.e., sequence of samples of the form [time, value]). Would it be possible to know in advance, based on the data only, if it makes sense to use a model for prediction (based on the samples, used as a training set).

I am asking this for the following simple reason: it is possible that the sample data is totally random and that there is no correlation at all between the samples. Hence, I would like to avoid trying to find correlations where there is not.

Answer 1

Why don't you omit a few data points, when working out the parameters of your model? Then see if the model correctly predicts the data points that you omitted. You'll need to decide ahead of time how accurately the model needs to predict the missing data point, in other words, what constitutes "success" of the model.

Answer 2

I lack a theoretical proof, however I strongly suspect that no such test exists.

My reasoning is that it is possible to design pseudo random number generators that meet strong statistical measures of randomness ( For example, the BSI tests. See wikipedia for details ). Nonetheless, such generators are deterministic and so, in principle, it should be possible to discover the underlying algorithm via machine learning ( consider, for example, a genetic programming system that manages to reproduce the generator exactly ).