(fixed problem statement) Complex conditional probability problem

Question

UPDATE: Sorry, I have just realised that my question was edited and it was completely wrong... I have reversed the edit and now it should make sense.

The problem is as follows:

There is a sticky ball that can be thrown to a glass window. Whenever we throw the ball at the glass it will stick for a number of seconds, with the following chances: $$ \begin{matrix} \mathrm{Time}\:(s) & \mathrm{EventWeight} & \mathrm{Chance = weight/sum}\\ 1 & 4 & 0.4 \\ 2 & 3 & 0.3 \\ 3 & 2 & 0.2 \\ 4 & 1 & 0.1 \\ \end{matrix} $$

($\mathrm{Time}=1$ means it will stick only for this second, so the next second it will have fallen). Whenever the ball is not sticking to the glass, there is chance:

Chance To (re)throw = 0.1

Now what I'm trying to calculate is:

During say $10$ seconds, at any one-second, what is the chance that the ball will be sticking on the glass? (that includes when it has just been thrown by you at the glass, or when it was sticking since a previous second).

For the first second, the chance is obviously 0.1 (chance to throw it), but for subsequent seconds it will start getting complicated because you don't know if it is still sticking from previous seconds or it has fallen and there is chance 0.1 to rethrow.

I made a simulation to verify, check the code at the end.**

---

Here is where I am now:

Say t denotes the current second.

I think you need to calculate the chance of the ball being on the glass for t=1, t=2, t=3,t=4 seconds separately then for the chance when t=5 to 10, it will be equal to the chance when t=4.

(Correction: Actually it starts repeating at t=5 according to the simulation, so I'm wrong here)

I can calculate for t=1 and t=2 easily, but once I reach t=3 it gets complicated.

My steps:

My steps:
t=1     Chance = 0.1

t=2     Chance = 0.1 + (chance of sticking from previous) - intersection of both events
            chance of sticking from previous = 0.1 * (chance of duration > 1) = 0.1 * 0.6 = 0.06
            = 0.1 + 0.06 - 0.1*0.06 = 0.154

t=3     Chance = 0.1 + (chance of sticking from previous) - intersection of both events:
            chance of sticking from previous = ?? how do you backtrack here ??

---

Code to get the answer computationally in Python:

from random import randint, random

def accumulate(l):
    sum = 0
    r = list()
    for i in l:
        sum += i
        r.append(sum)
    return r

class WeightedList(object):
    def __init__(self, values, weights):
        self.values = values
        self.weights = list(accumulate(weights))

    def get(self):
        r = randint(0, self.weights[-1] - 1) # exclude last
        for i in range(len(self.values)):
            if r < self.weights[i]:
                return self.values[i]

n = 10
d = WeightedList([1,2,3,4],[4,3,2,1])
ch = 0.1
trials = 1000000

logStick = [0.0] * n # preallocate n places

for i in range(trials):
    sticking = False
    remains = 0
    for j in range(n):
        if remains == 0:
            sticking = False

        if not sticking:
            if (random() < ch):
                sticking = True
                remains = d.get()

        logStick[j] += 1 if sticking else 0
        remains-=1

for i in range(n):
    logStick[i] /= trials
print(logStick)

Answer is:

Chance of the ball being sticking for each second:

    [0.100156, 0.15379, 0.178065, 0.183273, 0.181768, 0.181618, 0.181672, 0.182047, 0.181956, 0.181613]

As you can see, my assumption that it will start repeating starting from t=4 was slightly wrong... it actually starts repeating after t=4, which means starting from t=5.

You can also see that my calculation for when t=2 seems to be close but not actually correct..

Answer 1

The answer is actually isn't as difficult as I thought, it's solved using normal Markov Chains:

First an example to make my question clear again:

t=1
0.1 chance to throw (assume you throw and it should stay for 2 seconds)
Now the ball is sticking
t=2
The ball is still sticking
t=3
The ball has fallen, there is 0.1 to throw it again IMMEDIATELY on this second

We have five states:

1. in hand
2. 1 second
3. 2 seconds
4. 3 seconds
5. 4 seconds

\begin{array} {ccc} 0.9 & 0.04 & 0.03 & 0.02 & 0.01 \\ 0.9 & 0.04 & 0.03 & 0.02 & 0.01 \\ 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ \end{array}

Rows: 1,2,3,4,5 top-down

Columns: 1,2,3,4,5 left-right

The only tricky part here (to me at least) was to see that the second row (so, for when state is 1 second left) should be:

\begin{array}{cc}0.9 & 0.04 & 0.03 & 0.02 & 0.01\end{array}

and not:

\begin{array}{cc}1 & 0 & 0 & 0 & 0\end{array}

Because the ball can be thrown immediately when it falls, so you can actually get from state 1 second to all other states and not only to "in hand" state.

That way we can calculate easily for any t = X:

v * M ^ X

Where

M = probability matrix

v = initial state vector [1,0,0,0,0]

And sum the first row excluding the first cell to get the probability being on the glass at t = X

The result we get for t = 1 .. 10 is:

\begin{array} 0.1 & 0.154 & 0.17776 & 0.1837144 & 0.18210434& 0.18178471& 0.18179264& 0.18181786& 0.1818193 & 0.18181838 \end{array}

Answer 2

As Space monkey figured out the transition matrix, it can be used to calculate the long-term stationary probabilities which are $(0.818181818, 0.090909091, 0.054545455, 0.027272727, 0.009090909)$ or as ratios $(90, 10, 6, 3, 1)$. The chance that it will be sticking on the glass at any time is the 1 - probability it is in your hand. The long-term probability for it being in your hand (state 1) is $.818181818181,\; (\frac{9}{11})$ so the probability that in the long term it is sticking on the glass is $.181818181818,\; (\frac{2}{11})$ As Space monkey showed above, it converges pretty quickly.

If I understand the Wikipedia entry, the speed of convergence depends exponentially on the ratio between the absolute value of the first two eigenvalues, which here is 9:1 about $15.8:1$, which may account for that speed.