I am given a two-state Markov chain with transition matrix
$$ \left( \begin{array}{cc} 1-p & p \\ 0 & 1 \end{array} \right) $$
and am asked to calculate the mean time of hitting state $2$ given that the chain starts at state $1$. Let $X_i$ be a R.V. which represents the state at time $i$, so $X_0 = 1$. Also, I let $F_n(x,y)$ be the probability that state $y$ is reached for the first time after $n$ steps if we start with state $x$.
I showed that $F_n(2, 1) = (1-p)^{n-1}p$ using the time invariance property of Markov Chains and recursion. Now I am asked to find the mean time of hitting state $2$ which by the definition of expectation I think should be
$$ \sum_{n=1}^\infty n(1-p)^{n-1}p. $$
However, I can't see how to find the sum of this series since it has an $n$ term that's growing linearly, but presumably the $(1-p)^{n-1}$ is decreasing as a power so the terms should go to $0$ sufficiently fast.
A generating function approach. Say that:
$$ F(x) = p\sum_{n=0}^\infty n(1-p)^{n-1}x^{n-1} $$
Note that I changed the limit from $n = 1$ to $n = 0$ and confirm this is valid. This makes for an easier application of the geometric series later. Then we have $F(1)$ as your wanted expected value ($xF(x)$ is the G.F. for the terms of your sum). Then, integrating both sides and rearranging:
$$\int F(x)dx = \frac{p}{1-p}\sum_{n=0}^\infty (1-p)^{n}x^n$$
Now we can use the formula for the geometric series:
$$\int F(x)dx = \frac{p}{1-p}\cdot \frac{1}{1-(1-p)x}$$
And we can differentiate again to get back $F$ (remember $\frac{d}{dx}(1-cx)^{-1} = c(1 - cx)^{-2}$):
$$F(x) =\frac{p}{((1-p)x - 1)^2}$$
Now substitute $F(1)$ to find expected value $\frac{1}{p}$.
The crux of the above answer is to use integration to solve $\sum nc^n$. We get rid of the factor $n$ by using $\int x^{n-1}dx = \frac{1}{n}x^n$.