I have been working on the following problem:
The transition matrix of a driver moving in zone 1 to zone 2 and between the zones is given by P= [0.8 0.2, 0.3 0.7]. The state space is {zone1, zone2} The driver’s expected profit for a trip entirely in zone one is 8 dollars, entirely in zone two is 10 dollars and 12 dollars between zones. What is the expected profit per trip?
I have calculated the stationary probability vector as π=(3/5 , 2/5) but i do not know how to relate this to the expected profit. Also i calculated P^20, which converges to [0.6 0.4 , 0.6 0.4].
I tried multiplying the long-term transition probability matrix above by the profits and adding them together but i got 20.8, which does not seem like a likely answer.
If I'm in state 1, the next trip will have average profit 8.8 (0.8 x 8 + 0.2 x 12). If I'm in state 2, the next trip will have average profit 11.2. The steady state values indicate that you are in state 1 about 60% of the time and in state 2 the other 40%, so I get 9.76 as the average profit.