Reference request for this topics

Question

I'm a second year undergraduate statistic student, I need a good reference to learn these topics

1. Markov Chains in discrete time.

      1.1. Classification of states, recurrence notions of transience.

      1.2. Stationary measure.

      1.3. Reversibility.

2. Markov Chains continuous time.

3. Poisson process.
4. Processes of birth and death.
5. Applications: renewal theory notions of queuing theory.

I've had a look and seem to have video classes at MIT, I appreciate any indication

Is there any significant difference between:

1. Sheldon M. Ross "Stochastic processes"
2. Sheldon M. Ross "Introduction to probability models"

EDIT:I'm sorry for reviving this question, but I've been trying to study through 2. Sheldon M. Ross "Introduction to probability models", but this book is very tiring to read, now I'm looking for lecture notes or a more concise book only for the theory without many examples, and use Ross's book only to solve the exercises, since this has solutions.

Has anyone used any of the books below for reference?

Essentials of Stochastic Processes

A First Course in Stochastic Processes

Answer 1

A good starting point would be Introduction to Probability Models by Sheldon Ross. I think it covers all the topics that you described. I like the book because it's easy to read and has plenty of problems to try out, which makes it ideal for self learning.

Answer 2

I can recommend the following book

"Probability and Statistics by Example: Volume 2, Markov Chains: A Primer in Random Processes and their Applications" by Yuri Suhov and Mark Kelbert

This book covers most of your topics (depends of course how deep you want to dive into), is well written and explains everything with examples and solved exercises. So at least definitely a good source for exercises.

A very good companion for further readings in Markov chains on general state spaces is this book

"S.P. Meyn and R.L. Tweedie (1993), Markov chains and stochastic stability"

you can access it here for free.

I wouldn't recommend it for you as a primary source since it doesn't match exactly your criteria, but it is a really well written book, a lot of motivational examples and covers quite some theory.

bests