Suppose that we are flipping coins iteratively, until we get tails two in a row. Define three states: Heads, Tails, and Finished. Suppose that the probability of getting a head is $p$, and probability of getting a tail is $(1-p)$.
Required: (i) Find the Stochastic Matrix $M$ that models this situation. Define the column as the state you are in currently, and the rows as the state you might transitioning to; (ii) Find the probability that you finish before the $4$th flip, assuming that the first flip is a head.
For the first question, am I getting it right? $$M=\begin{align} \begin{bmatrix} p & p & 0 \\(1-p) & 0 & 0 \\0 & (1-p) & 1 \end{bmatrix} \end{align}$$
I have no idea how to do the second question. Any help would be greatly appreciated.
(i) I would transpose the matrix you have. Having all rows adding to $1$ is the standard format for stochastic matrices as far as I'm aware.
EDIT: As noted below by Ian, if you're coming from a linear algebra standpoint, your matrix is fine as it is.
(ii) Finishing before the 4th flip assuming the first is a head just means flips $2$ and $3$ must be tails, which has probability $(1-p)^2$ and doesn't need Markov Chains to solve. So the question seems odd in the context of Markov Chains - are you sure you have written it correctly?
Anyway, using the Markov Chain method, the required probability is that of moving from State $0$ to State $2$ in at most $2$ steps (the given assumption puts us in State $0$ after the first step). Since State $2$ is an absorbing state, this means we need $p_{02}^{(2)}$ and this value will appear in matrix $M^2$, and it is $(1-p)^2$ as given above.