Exercise :
A particle is moving randomly on the $x$-axis. At each instance of time, it moves either right with probability of $p$ or left with probability $1-p$. Each step of movement is independent of the rest. Initially, the particle lies at $0$.
(a) Calculate the probability of the particle being at $0$ after two instances of time.
(b) Calculate the probability of the particle being $1$ step to the right of $0$ after three instances of time.
(c) Given that after three instances of time the particle was $1$ step to the right of $0$, what is the probability of it being $1$ step next to $0$ at the first time instance ?
Attempt :
(a) There are two cases to take into consideration. During the first instance of time, the particle either moved right with $p$ or left with $1-p$.
Let $Z$ be the event of the particle being at $0$ after two instances of time. Let $R$ be the event of the particle going $1$ step right of $0$ during the first instance of time and $L$ to the left.
Then, by the Total Probability Theorem, it should be :
\begin{align*}\mathbb{P}(Z) &= \mathbb{P}(R)\mathbb{P}{(Z|R)} + \mathbb{P}(L)\mathbb{P}(Z|L)\\ &= p(1-p) + (1-p)p \\ &= 2p(1-p)\end{align*}
(b) Now there are more cases to consider.
First case : The particle moved one step left to $-1$ at the first instance of time, thus it must move two steps to the right to be at $1$ at the third instance of time.
Second case : The particle moved to $1$ at the first instance of time, then to $0$ at the second instance of time and then again to $1$ at the third instance of time.
Third case : The particle moved to $1$ and then to $2$, then back to $1$.
For the first two cases, it comes down to having the probability of the particle moving to $0$ after two instances of time, which we do have from (a).
Let $E$ be the event of the particle residing at $1$ after three instances of time. Then again by the Total Probability Theory, it should be :
\begin{align*} \mathbb{P}(E) &= \mathbb{P}(Z)\mathbb{P}(E|Z) + \mathbb{P}(RR)\mathbb{P}(E|RR) \\ &=2p(1-p)\cdotp p + p^2(1-p) \end{align*}
I used the fact that each step is independent from the rest, thus the probability of going $2$ steps to the right is $p \cdot p = p^2$.
(c) Let $F$ be the probability of the particle being $1$ step next to $0$ after the first time instance. We want to calculate the probability $\mathbb{P}(F|E)$, which by the Bayes Theorem, equals :
$$\mathbb{P}(F|E) = \frac{\mathbb{P}(E|F)\mathbb{P}(F)}{\mathbb{P}(E)} = \frac{\mathbb{P}(E|F)\cdot p}{2p(1-p)\cdotp p + p^2(1-p)}$$
We calculate $\mathbb{P}(E|F)$ similarly with $b$.
QUESTION : Is my elaboration on all these steps correct ? Am I missing something in my thought process ? I would very much appreciate any input.
Your solution seems correct, although I think the way you formalize the problem could be improved. This allows you to answer virtually any question regarding this problem.
Let $x_t$ be the position of the particle at time $t$. We know that:
$$x_0 = 0,$$ $$P(x_t|x_{t-1}) = \begin{cases} p, &\text{ if } x_t = x_{t-1}+1,\\ 1-p, &\text{ if } x_t = x_{t-1}-1,\\ 0, &\text{ otherwise.} \end{cases} \,\,\,\text{ (for }t \geq 1),\text{ }$$ $$(x_t \perp x_0, x_1, \dots, x_{t-2})| x_{t-1}.$$
(The last is a notation for saying that $x_t$ is independent of all past positions $x_0, x_1, \dots, x_{t-2}$ given the previous position $x_{t-1}$.)
Now let's use this systematically in a concrete example. Let's determine the probability that the particle was one step left from the origin at the first time step, provided that it was one step right from the origin at the third time step. Thus, we want to determine $P(x_1=-1 | x_0=0, x_3=1)$. By the Bayes' theorem, we have:
\begin{align} P(x_1=-1 | x_0=0, x_3=1) &= \frac{P(x_1=-1, x_3=1|x_0=0)}{P(x_3=1|x_0=0)} \\&= \frac{P(x_3=1|x_1=-1,x_0=0)P(x_1=-1|x_0=0)}{P(x_3=1|x_0=0)}. \end{align}
Note that, due to the conditional independence, we have:
$$P(x_3=1|x_1=-1,x_0=0) = P(x_3=1|x_1=-1),$$
and this can be computed easily by marginalizing $P(x_3=1, x_2|x_1=-1)$ with respect to $x_2$:
\begin{align} P(x_3=1|x_1=-1) &= \sum_{x_2}P(x_3=1, x_2|x_1=-1) \\ &= \sum_{x_2} P(x_3=1| x_2, x_1=-1) P(x_2|x_1=-1) \text{ (Bayes' theorem)} \\ &= \sum_{x_2} P(x_3=1| x_2) P(x_2|x_1=-1) \text{ (condit. indep.)} \\ &= P(x_3=1 | x_2=0) P(x_2=0 | x_1=-1) \text{ (other cases have zero probability)} \\ &= p^2. \end{align}
We clearly have $P(x_1=-1|x_0=0) = 1-p$, so all that remains is computing $P(x_3=1|x_0=0)$:
\begin{align} P(x_3=1|x_0=0) =& \sum_{x_1,x_2} P(x_3=1, x_2, x_1|x_0=0) \\ =& \sum_{x_1,x_2} P(x_3=1|x_2) P(x_2|x_1) P(x_1|x_0=0) \text{ (Bayes' theorem + condit. indep.)} \\ =& P(x_3=1|x_2=0) P(x_2=0|x_1=-1) P(x_1=-1|x_0=0) \\ &+ P(x_3=1|x_2=0) P(x_2=0|x_1=1) P(x_1=1|x_0=0) \\ &+ P(x_3=1|x_2=2) P(x_2=2|x_1=1) P(x_1=1|x_0=0) \\ &\text{ (other cases have zero probability)} \\ =& p^2(1-p) + p(1-p)p + (1-p)p^2 \\ =& 3p^2(1-p). \end{align}
Thus,
$$P(x_1=-1 | x_0=0, x_3=1) = \frac{p^2(1-p)}{3p^2(1-p)} = \frac{1}{3}.$$
EDIT: Note that exercise (c) asks for the probability of the particle "being 1 step next to 0 at the first time instance" (i.e. $x_1 \in \{-1,1\}$), but this is trivial. We immediately have $P(x_1 \in \{-1,1\} | x_0=0, x_3=1)=1$, because if $x_0=0$ we necessarily have either $x_1=-1$ or $x_1=1$ (the particle either moved left or right).
As an exercise, you may want to compute $P(x_1=1 | x_0=0, x_3=1)$ as we did for the other case and check that it equals 2/3, so that \begin{align} P(x_1 \in \{-1,1\} | x_0=0, x_3=1) &= P(x_1=-1 | x_0=0, x_3=1) + P(x_1=1 | x_0=0, x_3=1) \\ &= \frac{1}{3} + \frac{2}{3} = 1. \end{align}