Self Learning Stochastic Process By Sheldon Ross

Question

I have just started self-learning Stochastic Processes by Sheldon Ross (2nd Edition). I am finding the exercises really tough and time-consuming. I even tried searching for a solution manual but couldn't find it anywhere.

I have gone through the book - A first course in Probability by the same author before. But, Stochastic Processes is on a completely different level for me. I wanted some advice from people who have learned from this book on how did they go about it and verify their solutions. If someone knows about the availability of a solution manual, could someone direct me towards the same?

Answer 1

What specifically are you having trouble with in Ross's Stochastic Processes? I am familiar with this text and I would have to say it has its shortcomings. Although the preface states

This text is a nonmeasure theoretic introduction to stochastic processes, and as such assumes a knowledge of calculus and elementary probability.

The first chapter begins with the formal measure-theoretic definition of a probability space, and proceeds to introduce and prove the Borel-Cantelli lemmas, which are statements about the $\limsup$ of a sequence of sets. It is unlikely the notion of limit superior would have been introduced in a typical undergraduate calculus and introductory probability courses; and it is not mentioned at all in First Course in Probability - so I could see how this maybe be confusing.

The concept of expectation is defined in terms of Riemann-Stieltjes integrals, as opposed to Lebesgue integrals, however, and indeed this is treated in 7.9 of the 10th edition of First Course in Probability. Also, conditional expectation is not defined rigorously in terms of the Radon-Nikodym theorem, so the "nonmeasure theoretic" claim is justified, I suppose.

Going chapter-by-chapter in Stochastic Processes:

1. Most of this material is treated in First Course in Probability. Characteristic functions and Laplace transforms are not, but the general concept is similar enough to that of the moment-generating function that it shouldn't be too hard to pick up on. The last section which formally introduces stochastic processes is of course fundamental to the rest of the text, and the definitions and examples seem reasonable to me.
2. First Course in Probability introduces the Poisson process in much the same manner as the beginning of this chapter. The remainder rigorously proves important and often-used properties of the Poisson process, although section 2.6 "Conditional Poisson Processes" is a bit more advanced (such processes are also known as doubly stochastic Poisson processes or Cox processes), and could probably be skipped on a first reading.
3. Renewal processes are a generalization of Poisson processes and are extremely important in the study of stochastic processes. A rigorous proof of the strong law of large numbers is given in First Course in Probability, and the techniques used there are important for being able to follow the proofs of the results in this chapter. But for example, stopping times are not defined formally in the measure-theoretic sense, nor is Wald's identity stated in full generality - so this should be understandable, with a bit of effort. The proof of the elementary renewal theorem is a bit more involved, though not beyond the level of an introductory real analysis course. The Key renewal theorem doesn't seem to be explained well, but the coverage on alternating renewal processes, the inspection paradox, delayed and equilibrium renewal processes, renewal reward processes, and regenerative processes all seem solid to me.
4. The chapter on Markov chains seems a bit lacking, only covering some of the more fundamental results and focusing heavily on examples which to me do not seem well-motivated. In particular semi-Markov processes seem a bit out of place in a first introduction to Markov Chains.
5. The chapter on continuous-time Markov chains is very thorough and does a good job of covering the essential topics, as well as providing important applications of queueing theory.
6. The topic of martingales really should be left to a second course in stochastic processes which assumes prior knowledge of measure theory, in my opinion. I would not recommend this text as an introduction to the topic.
7. The general random walk is another topic for which "a knowledge of calculus and elementary probability" is insufficient, in my opinion.
8. Brownian motion is an extremely important topic in the study of stochastic processes...but I really don't think it is treated well in this text or given the proper motivation. The construction of the process is not compelling, the reflection principle does not seem to be mentioned, and the pathological properties of sample paths (i.e. continuous with probability one but differentiable nowhere) not discussed either. 9-10. These chapters are a bit advanced for an introductory course, in my opinion.

Ross's book does have good exercises and many examples, but I find the theory and motivation a bit lacking. I might suggest Resnick's Adventures in Stochastic Processes instead.

If you have any specific questions about the material feel free to elaborate, of course.

Answer 2

I am also working on this book. Here are the websites I frequently visit for hints when doing exercises.

http://www.charmpeach.com/stochastic-processes/solutions-to-stochastic-processes-ch-4/723/

http://www.columbia.edu/~ww2040/6711F13/homework6711.html

Hope this helps.