Correlation in a series of 1s and 0s

Question

I have a series (in reality I have several series) of 1s and 0s (success and failures) and an "estimated" success-probability as well as the actual success-rate (I can count the 1s in my set).

I want to see if there is any correlation between if the previous outcome was a success and the success of the next outcome (also if the outcome have any affect on the outcome of the lets say next 3 outcomes).

I've tried asking multiple times here but it seems I am asking the wrong question because no one is answering me but instead tells me to "look at research" or that "I am just stupid".

Please, anyone have any ideas how I would go about finding any dependence between the outcomes?

Any ideas are welcome but no "This is a well studied subject please read this" as I have most likely all ready read it and still cant understand how to go about it.

Ive looked into auto-correlation but honestly cant grasp how to apply it to my problem.

Thank you in advance

Answer 1

Maybe it will help if we compare a sequence of 0's and 1's that is not independent with another sequence that is.

Dependent (weather model). A Markov chain involves modeling one of the simplest kinds of dependence. Each value can depend on the previous one, but knowing the previous value there is no advantage in knowing earlier ones.

A simple example is an imaginary island where weather can be sunny (0) or rainy (1) on any one day according to these rules: If it is rainy today, it will be rainy tomorrow with probability .7. If it is sunny today, it will be sunny tomorrow with probability .7. One can show that over the long run it is rainy on half of the days.

Here is a short R program that will generate such a random sequence for 1000 days. Over our simulated 1000 days, we get barely less than half rainy days, specifically 477,

 m = 1000;  x = numeric(m);  x[1] = 0
 for (i in 2:(m-1)) {
   if (x[i] == 0) {x[i+1] = rbinom(1, 1, .3)}
   else x[i+1] = rbinom(1, 1, .7)
 }
 mean(x)
 ##  0.477

The first plot below shows 0's and 1's for the first 20 days. Notice the pattern: rainy days and sunny days tend to occur is sequential 'clumps' of a few days, rather than alternating independently. The second plot is an $autocorrelation\; function$ (ACF) plot for lags up to 30. The ACF plot shows that once a rainy or sunny pattern starts, it often takes a few days for the pattern to dissipate.

The ACF for 'order' or 'lag' $1$ is essentially the correlation of the sequence $X_1, X_2, \dots, X_{999}$ and the sequence $X_2, X_3, \dots, X_{1000}$ This is not 'exactly' the correlation of the two sequences because the sample means and SDs in the correlation formula all use all 1000 observations instead of 999. You should look for elementary discussions of 'autocorrelation' online for more details.

The first autocorrelation in the plot is of order 0 (no lag). It is always 1. (A correlation of $X_1, X_2, \dots, X_{1000}$ with itself.)

There is a test for when an autocorrelation of a particular lag is significantly different from $0.$ The band marked with blue dashed lines in the ACF plot indicate values that are consistent with zero autocorrelation.

Independent (coin tossing). By contrast, here is corresponding output for a sequence of independent coin tosses, with results Heads (1) and Tails (0). Here the first plot shows changes independently and at random between values 0 and 1. Alternation between 0 and 1 tends to be a little more rapid; but, of course, repeated values are possible.

 x = rbinom(1000, 1, .5)
 mean(x)
 ## 0.507

 ## code for plot
 par(mfrow=c(1,2))  # enables two panels per plot
 plot(x[1:20], type="b", pch=20)  # "b" means Both points and lines
 acf(x)             # acf plot
 par(mfrow=c(1,1))  # return to single-panel plots

Note: There are a lot of concepts lurking in the questions you are asking. I have tried to illustrate a few that may help get you started. Please do not get discouraged by comments that seem snarky to you. One step at a time, and you'll get the degree of understanding you want.