Question

Does this proof in Puterman's MDP book make sense?

score 62 · Answer 1 · 2026-03-25 14:20:56

62

Views

Does this proof in Puterman's MDP book make sense?

Published on 25 Mar 2026 - 14:20

score 49 · Answer 2 · 2026-03-25 12:38:44

49

Views

Given a Markov process, are we able to construct another Markov process with the same transition semigroup but different inital law?

Published on 25 Mar 2026 - 12:38

score 56 · Answer 3 · 2026-03-26 14:18:44

56

Views

understanding Krylov-Bogolubov Theorem

Published on 26 Mar 2026 - 14:18

score 36 · Answer 4 · 2019-05-23 09:41:16

36

Views

Markov property for Simple Birth Process at random time

Published on 23 May 2019 - 9:41

score 142 · Answer 5 · 2019-05-25 07:49:21

142

Views

A aperiodic state of a Markov chain has $N\geq 1$ such that $\forall n\geq N:p_{i,i}(n)>0$

Published on 25 May 2019 - 7:49

score 159 · Answer 6 · 2019-05-27 19:52:22

159

Views

Conditions for which groups of states of markov chain still represent a markov chain

Published on 27 May 2019 - 19:52

score 501 · Answer 7 · 2026-02-23 06:25:21

501

Views

Show a markov chain with transition matrix $P$ and a markov chain with matrix $\frac{1}{2}(I+P))$ have the same invariant distribution

Published on 23 Feb 2026 - 6:25

score 102 · Answer 8 · 2026-03-25 12:38:54

102

Views

Trotter-Kato approximation theorem for transition semigroups of Feller processes

Published on 25 Mar 2026 - 12:38

score 246 · Answer 9 · 2026-02-23 06:21:00

246

Views

Why is the following a Markov Chain?

Published on 23 Feb 2026 - 6:21

score 64 · Answer 10 · 2019-05-31 13:53:53

64

Views

Differential equation for a probability density

Published on 31 May 2019 - 13:53

score 35 · Answer 11 · 2019-06-01 15:33:59

35

Views

Probability that Markov chain state is not touched for at least n steps

Published on 01 Jun 2019 - 15:33

score 131 · Answer 12 · 2019-06-01 17:14:01

131

Views

Slot machines fundamental matrix interpretation

Published on 01 Jun 2019 - 17:14

score 336 · Answer 13 · 2019-06-01 17:18:14

336

Views

How to find the fraction of time spent in each state of a periodic Markov chain

Published on 01 Jun 2019 - 17:18

score 78 · Answer 14 · 2019-06-03 02:16:52

78

Views

What is the probability that Pat wins?

Published on 03 Jun 2019 - 2:16

score 93 · Answer 15 · 2026-03-25 12:41:38

93

Views

If $X$ is a Feller process, then $\sup_{x\in E}\text E\left[d(X_s,X_t)\wedge1\mid X_0=x\right]\xrightarrow{s-t\to0}\to0$

Published on 25 Mar 2026 - 12:41

List Question

Does this proof in Puterman's MDP book make sense?

Given a Markov process, are we able to construct another Markov process with the same transition semigroup but different inital law?

understanding Krylov-Bogolubov Theorem

Markov property for Simple Birth Process at random time

A aperiodic state of a Markov chain has $N\geq 1$ such that $\forall n\geq N:p_{i,i}(n)>0$

Conditions for which groups of states of markov chain still represent a markov chain

Show a markov chain with transition matrix $P$ and a markov chain with matrix $\frac{1}{2}(I+P))$ have the same invariant distribution

Trotter-Kato approximation theorem for transition semigroups of Feller processes

Why is the following a Markov Chain?

Differential equation for a probability density

Probability that Markov chain state is not touched for at least n steps

Slot machines fundamental matrix interpretation

How to find the fraction of time spent in each state of a periodic Markov chain

What is the probability that Pat wins?

If $X$ is a Feller process, then $\sup_{x\in E}\text E\left[d(X_s,X_t)\wedge1\mid X_0=x\right]\xrightarrow{s-t\to0}\to0$

Trending Questions

Popular # Hahtags

Popular Questions