I am counting streaks that occur in financial data (consecutive positive increases or decreases). I want to know with what certainty a streak is likely to continue once it occurs and continues to occur. For example, one part of my data looks like streaks of
5: 84
of 6: 52
of 7: 28
of 8: 11
of 9: 5
of 10: 2
of 12: 2
Total 184 streaks.
So how do I tell with what confidence&probability (I dont really know the difference in this case and would like to know both) a streak of 5 will continue to 6, 6 to 7, etc. How can I get these estimates for streaks that didn't occur like of 11 or 13?
Thank you for your help!
Computing confidence or probabilities requires choosing a model for your data. There are many ways you could do this, each involving different levels of difficulty in estimation and assumptions; for instance, whether you want to use a time series method to model the underlying "up vs down process" versus trying to fit a probability distribution to try to model the streaks and their size itself. (The former case makes it easy to compute the chance of streaks continuing; the latter is what would be necessary to compute confidences.)
Anyway, here is a simple idea: use a Markov chain. A common assumption here is time-homogeneity, which says that the distributions governing the process do not change over time. An order $k$ Markov chain means that the value of the process at time $t$ depends only on the last $k$ values. Specifically, for example: \begin{align*} P(X_t=x_t|X_{t-1} = x_{t-1},\ldots,X_0=x_0) &= P(X_t=x_t|X_{t-1}=x_{t-1},\ldots,X_{t-k}=x_{t-k}) \end{align*} where $x_i=U$ means it moved up and $x_j=D$ means it moved down.
You then have to fit your Markov chain to the data (i.e. estimate the various transition probabilities).
For instance, for $k=1$, the transition probabilities can be written as a matrix: $$ P = \begin{bmatrix} P(X_t=U|X_{t-1}=U) & P(X_t=D|X_{t-1}=U)\\ P(X_t=U|X_{t-1}=D) & P(X_t=D|X_{t-1}=D) \end{bmatrix} $$ In this highly simplified case, one can compute the probability of an "up" streak continuing as just $P(X_t=U|X_{t-1}=U)$.
You can then compute the probability of a streak occurring directly as well, e.g. a streak of size 3, starting from $ X_t=D $, as $$P(X_{t+3}=U|X_{t+2}=U)\,P(X_{t+2}=U|X_{t+1}=U)\,P(X_{t+1}=U|X_{t}=D)$$
Also, since you are using financial data, you might want to consider continuous time stochastic process models instead. E.g. for stock prices a simple model is geometric Brownian motion, and there are more sophisticated ones (e.g. see here).