I am thinking about the following problem and I faced a minor issue:
Consider a bowl that contains initially two blue balls and one yellow ball. At every step you draw a ball randomly. The ball is then put back, and another ball of the same colour is added to the bowl before the next draw. Assume that the experiment is stopped after 2 draws, when the bowl contains 5 balls in total. Upon stopping, the bowl is emptied and the experiment is restarted again with two blue balls and one yellow ball in the bowl. What is the transition matrix that models the content of the bowl over time?
It seems pretty straight forward that the state space is {BBY, BBBY, BBYY, BBBBY, BBBYY, BBYYY} where B denotes a blue ball and Y denotes a yellow ball. However, I am confused about the probability for the transitions BBBBY -> BBY, BBBYY -> BBY and BBYYY -> BBY. Should these equal 0, or 1 since this is a repeated experiment? Thanks a lot.
Since there is no end to the stochastic process, our transition matrix will model the resetting process. In pre-multiply form it is $$A=\begin{bmatrix} 0&0&0&1&1&1\\ 2/3&0&0&0&0&0\\ 1/3&0&0&0&0&0\\ 0&3/4&0&0&0&0\\ 0&1/4&1/2&0&0&0\\ 0&0&1/2&0&0&0 \end{bmatrix}$$ This is the pre-multiply form because the probability of being in a certain state after $n$ steps can be found as $$\begin{bmatrix}p_{BBY}\\p_{BBBY}\\p_{BBYY}\\p_{BBBBY}\\p_{BBBYY}\\p_{BBYYY}\end{bmatrix}=A^n\begin{bmatrix}1\\0\\0\\0\\0\\0\end{bmatrix}$$