Let $\xi_n$, n $\in \mathbb{Z_+}$ be the sequence of independent random variables defined on $\mathbb{R}$ with common density function $p(x) > 0$. How to know the following sequences are Markov or not? If the answer is "yes" find transition probabilities.
a) $\eta_n := \sum_{k=1}^{n}\xi_k$
b) $\eta_n := \sum_{k=1}^{n}\gamma^n\xi_k$, with $\gamma \in \mathbb{R}$
The first item looks like one-dimensional random walk. Since $\eta_n = \eta_{n-1} +\xi_n$ it is easy to understand that the sequence is Markov just by the defenition (look). I tried to calculate transition probabilities using distribution function:
$Pr(\eta_{ n+1}< x |\eta_n = b)=\int_{-\infty}^{x-b}p(s)ds$
Is it a correct answer?
The second item is very similar to the first one, here we can express $\eta_n = \gamma\eta_{n-1} +\gamma^n\xi_n$, therefore it is also Markov seguence. So the answer here is
$Pr(\eta_{ n+1}< x |\eta_n = b)=\int_{-\infty}^{x/\gamma^{n+1}-b/\gamma^{n}}p(s)ds$
My main question is how to generalize these answers, in other words how can I get $Pr(\eta_{n+s}< x |\eta_n = b)$ for any possible $s$?
We first figure out the expression for $P(\eta_{n+s}<x \mid \eta_n=b)$, when $s=2$ and $\eta$ is as in case (a).
Note that, as $\xi_n$'s are identically distributed, $P(\eta_{n+2}<x\mid \eta_n=b) = P(\eta_2 =\xi_0+\xi_1 +\xi_2<x| \eta_0 =\xi_0=b) = P(\xi_1 +\xi_2 <x-b)$.
And for any $a \in \mathbb{R}$, $$ P(\xi_1 + \xi_2 < a) = \int_{-\infty}^{\infty}P(\xi_1 < y, \xi_2<a-y)\,d y \\= \int_{-\infty}^{\infty}P(\xi_1 < y)P( \xi_2<a-y)\,d y \\= \int_{-\infty}^{\infty} \left(\int_{-\infty}^{y}p(x)\,d x\right)\left(\int_{-\infty}^{a-y}p(z)\,d z\right)\,dy.$$ This is as far as one can get with the information given in the question. Of course, you can replace $a$ by $x-b$ in the above to get your desired result.
For more general $s \in \mathbb{N}$, an expression such as the one above will be quite complicated. However, because of the independence of $\xi_n$'s, in this case you can use the fact that if $\xi_n$'s have (common) probability distribution $P_{\xi}$, then the probability distribution of $\eta_n $ will be $(P_{\xi})^{*n}$, the $n$-times convolution product of $P_{\xi}$'s.