Suppose that $X_n$ is a Markov chain. Let $S_j = \text{Number of periods that $X_n = j$}$. Let $P_{ij} = P(X_1 = j | X_0 = i)$. Define $s_{ij} = E(S_j|X_0 = i)$ I want to prove that: $$ s_{ij} = \delta_{ij} + \sum_k P_{ik}s_{kj}, $$ where $\delta_{ij}$ denote the Kronecker delta. The problem is that I wind up with a different result, and I cannot find any problems with my proof:
Proof: We see that $E(S_j | X_0 = i, X_1 = j) = 1 + E(S_j | X_0 = j)$ $(\ddagger)$. That means:
$$ s_{ij} = E(S_j | X_0 = i) = \sum_{k} E(S_j | X_0 = i, X_1 = k)P(X_1 = k|X_0=i) \\ =\underbrace{E(S_j | X_0 = i, X_1 = j)}_{=1 + E(S_j | X_0 = j) \ \ (\ddagger)}P(X_1 = j|X_0=i) + \sum_{k \neq j} \underbrace{E(S_j | X_0 = i, X_1 = k)}_{=E(S_j | X_0 = k)}P(X_1 = k|X_0=i) = P(X_1 = j|X_0 = i) + \sum_{k} s_{kj}P(X_1 = k|X_0=i) = P_{ij} + \sum_{k} P_{ik}s_{kj} \neq \delta_{ij} + \sum_{k } P_{ik}s_{kj} $$
My question: How can I rectify my reasoning such that it becomes correct? I cannot find any problems with my current proof.
Shouldn't it be $$E[S_j|X_0=i,X_1=j]=\delta_{ij} + E[S_j|X_1=j] $$ where the conditional expectation on the RHS follows from the Markov assumption?
I suppose that it is a homogenous Markov chain and would proceed directly as follows, $$E[S_j|X_0=i]=\sum_k E[S_j|X_0=i,X_1=k]P_{ik} = \delta_{ij}+ \sum_k E[S_j|X_1=k]P_{ik} $$ where the second equality follows from the Markov property (conditioning on $X_1$) and from accounting that we might already start at $i=j$. Conclude by using the definition of $s_{kj}$ and the homogeneity.
Edit: we may use the time shift operator $\theta_\tau$ and write the second equality as $$E[S_j|X_0=i]= \ldots = \delta_{ij}+ \sum_k E[S_j\circ \theta_1 |X_0 \circ \theta_1=k]P_{ik}$$ and then use the 'homogenity Markov' argument to substitute the $s_{kj}$ for the expectation with the shift operator.