Starting at $n-1$ and proceed backwards, the update steps for the Rauch–Tung–Striebel smoother are (see RTS smoother on Wikipedia):
$\textbf{C}_{k} = \textbf{P}_{k|k}\textbf{F}_{k+1}^{T}\textbf{P}_{k+1|k}^{-1}$
$\hat{\textbf{x}}_{k|n} = \hat{\textbf{x}}_{k|k} + \textbf{C}_{k}(\hat{\textbf{x}}_{k+1|n} - \hat{\textbf{x}}_{k+1|k}) $
$\textbf{P}_{k|n} = \textbf{P}_{k|k} + \textbf{C}_{k}(\textbf{P}_{k+1|n} - \textbf{P}_{k+1|k})\textbf{C}_{k}^{T}$
I am having trouble understanding how this would be initialized in practice. My question is, what is the value of $\hat{\textbf{x}}_{n|n}$? If I set $\hat{\textbf{x}}_{n|n}$ to the last value of the forward pass, $\hat{\textbf{x}}_{n|n} = \hat{\textbf{x}}_{n|k}$, then $(\hat{\textbf{x}}_{k+1|n} - \hat{\textbf{x}}_{k+1|k})=0$ and $\hat{\textbf{x}}_{k|n}$ would always be equal to $\hat{\textbf{x}}_{k|k}$ which is obviously wrong. What am I missing here?
edit
I got confused by the notation. $\hat{x}_{n|n}$ is initialized to the posterior, so there is no issue since $\hat{\textbf{x}}_{k+1|k}$ refers to the a priori estimate.