I am struggling with how to calculate the values of a Markov matrix which has multiple states.
For example,
Imagine an unfair 6 sided dice. The chance of rolling a 1,2,3,4,5 or 6 is
0.3, 0.25, 0.2, 0.12, 0.10, 0.03 respectively
The idea is that you keep rolling the dice until the accumulated total is greater than 6. That gives us 7 states, 1,2,3,4,5,6,7+
This gives me the following transition matrix
0.3, 0
0.25, ?
0.2, ?
0.12
0.10
0.03
0
The first column in the Markov matrix is simply the probabilities listed above along with 0 for the chance of rolling greater than a 6 on first throw. The next column obviously starts with 0 because it is not possible to still have an accumulated chance of 1 after 2 throws. After is where I get confused. I have no idea how to calculate the probability of being in state 2 after the second throw because this relies on current state being a 1 and then throwing another 1 (surely it isn't simply 0.3). Even more confusing to me is the probability of being in state 3. This means the current state either has to be 1 and roll a 2 or current state 2 and roll a 1.
If anyone could explain to me how to calculate the probability of state 2 in column 2 and state 3 in column 2 I would greatly appreciate it.
Thanks for your time.
A Markov transition matrix contains the one-step transition probabilities from each state to every other state. With seven states your matrix is $7\times 7$, and if I have understood your process correctly, looks like:
$$\matrix{ &\color{red} 1&\color{red}2&\color{red}3&\color{red}4&\color{red}5&\color{red}6&\color{red}{7+}\cr \color{red}1& 0&.30&.25&.20&.12&.10&.03\cr \color{red}2& 0&0&.30&.25&.20&.12&.13\cr \color{red}3& 0&0&0&.30&.25&.20&.25\cr \color{red}4& 0&0&0&0&.30&.25&.45\cr \color{red}5& 0&0&0&0&0&.30&.70\cr \color{red}6& 0&0&0&0&0&0&1\cr \color{red}{7+}& 0&0&0&0&0&0&1\cr }$$
Note: The red labels on the rows and columns are not matrix values.