Suppose a sequence of random bits $(X_1,X_2,...)$ is defined as follows, where $X_0=1$ and $p\in(0,1):$ $$X_i=\begin{cases}X_{i-1}&\text{ with probability }p,\\[1ex] 1-X_{i-1}&\text{ with probability }1-p. \end{cases}\\[2ex]$$ Thus each random bit is either a direct copy or an inverted copy of its predecessor.
Question: How to prove the following? ... $$\text{For all }p\in(0,1),\ \lim_\limits{n\to\infty}P(X_1=1\mid X_n=1)=p.\tag{1}$$
Plots suggest conjecture (1), with monotonic convergence when $p\ge 1/2$ and oscillatory convergence when $p<1/2$:
For each $n$, $\mathsf P(X_1=1\mid X_n=1)={\mathsf P(X_1=1,X_n=1)\over\mathsf P(X_n=1)}$ is a ratio of polynomials in $p$, e.g. as below (in lowest form) for a few cases:
n P(X_1 = 1 | X_n = 1)
-- --------------------
1 1
2 p^2/(2*p^2 - 2*p + 1)
3 (2*p^2 - 2*p + 1)/(4*p^2 - 6*p + 3)
4 (4*p^4 - 6*p^3 + 3*p^2)/(8*p^4 - 16*p^3 + 12*p^2 - 4*p + 1)
5 (8*p^4 - 16*p^3 + 12*p^2 - 4*p + 1)/(16*p^4 - 40*p^3 + 40*p^2 - 20*p + 5)
6 (16*p^6 - 40*p^5 + 40*p^4 - 20*p^3 + 5*p^2)/(32*p^6 - 96*p^5 + 120*p^4 - 80*p^3 + 30*p^2 - 6*p + 1)
7 (32*p^6 - 96*p^5 + 120*p^4 - 80*p^3 + 30*p^2 - 6*p + 1)/(64*p^6 - 224*p^5 + 336*p^4 - 280*p^3 + 140*p^2 - 42*p + 7)
8 (64*p^8 - 224*p^7 + 336*p^6 - 280*p^5 + 140*p^4 - 42*p^3 + 7*p^2)/(128*p^8 - 512*p^7 + 896*p^6 - 896*p^5 + 560*p^4 - 224*p^3 + 56*p^2 - 8*p + 1)
If $n$ is not too large, these ratios are readily obtained by representing the events $(X_1=1,X_n=1)$ and $(X_n=1)$ as length-$n$ pattern strings of form $1??...?1$ and $???...?1$, respectively. Each of these can then be converted to a list of all possible bitstrings matching the pattern, whose probabilities are simple products whose sum is to be taken. E.g., with $q=1-p$, $$\mathsf P(X_1=1\mid X_n=3)={\mathsf P(1?1)\over \mathsf P(??1)}={\mathsf P(101)+\mathsf P(111)\over \mathsf P(001)+\mathsf P(011)+\mathsf P(101)+\mathsf P(111)}={pqq+ppp\over qpq+qqp+pqq+ppp}={2p^2 - 2p + 1\over 4p^2 - 6p + 3}.$$
We note that, apparently, when the ratios are expressed in lowest form, $$\text{numerator of $\mathsf P(X_1=1\mid X_n=1)$}=\begin{cases}\text{denominator of $\mathsf P(X_1=1\mid X_{n-1}=1)$}&\text{for odd }n>1\\[2ex] p^2\times\text{denominator of $\mathsf P(X_1=1\mid X_{n-1}=1)$}&\text{for even }n>1 \end{cases} $$
but I don't know if this is useful. (The question seems so elementary, I feel like a proof ought to be likewise -- sorry if I'm just not seeing something obvious.)
Let $Z_1, Z_2, \ldots$ be i.i.d. $\text{Bernoulli}(1-p)$. Let $X_i$ be the indicator for the event $\{Z_1 + \cdots + Z_i \text{ is even}\}$. Then the process $X_1, X_2,\ldots$ has the same distribution as what you have specified.
Then $$P(X_1=1, X_n=1) = P(Z_1=0, Z_2 + \cdots + Z_n \text{ is even}) = P(Z_1=0) P(Z_2 + \cdots + Z_n \text{ is even})\\ \approx p/2.$$ Similarly, $$P(X_1=0, X_n=1) = P(Z_1=1, Z_2 + \cdots + Z_n \text{ is odd}) = P(Z_1=1) P(Z_2 + \cdots + Z_n \text{ is odd}) \approx (1-p)/2.$$
So, we have $P(X_1=1 \mid X_n=1) \approx \frac{p/2}{p/2 + (1-p)/2} = p$.
To make the "$\approx$" rigorous, you essentially have to argue that "the probability that a $\text{Binom}(n, 1-p)$ random variable is even tends to $1/2$ as $n \to \infty$"; I have hand-waved it away for the sake of giving you intuition.