I am trying to calculate the distribution of some univariate variable $X$ that follows a Normal distribution with mean $\mu$ but $\mu$ itself is also random. For simplicity I assume Normal distributions with the same known variance. Hence:
$$ p(x | \mu) = \frac{1}{\sqrt {2 \pi \sigma^2}} \exp-\frac{(x-\mu)^2}{2 \sigma^2}$$
and
$$ p(\mu | x_o) = \frac{1}{\sqrt {2 \pi \sigma^2}} \exp-\frac{(\mu - x_0)^2}{2 \sigma^2}$$
Therefore:
\begin{align} p(x) & = \int p(x|\mu) \, \,p(\mu|x_0) \, d\mu \\\\ & = \frac{1}{2\pi\sigma^2} \int \exp\big[-\frac{1}{2\sigma^2}\big( (x-\mu)^2 + (\mu-x_0)^2 \big)\big]d\mu \end{align}
Expanding now the demoninator inside the exponential function we get \begin{align} (x-\mu)^2 + (\mu-x_0)^2 & = x^2 + \mu^2 - 2 x \mu + \mu^2 + x_0^2 - 2 x_0 \mu \\\\ & = 2\mu^2 -2\mu(x + x_0) +x^2+x_0^2 \\\\ & = 2\mu^2 -2\mu(x + x_0) +x^2+x_0^2 + 2\, x\,x_0 - 2\, x\,x_0 \\\\ & = 2\mu^2 -2\mu(x + x_0) +(x+x_0)^2 - 2\, x\,x_0 \\\\ & = \mu + (\mu-(x+x_0))^2 - 2\, x\,x_0 \end{align}
Can I work this any further please and get the distribution of $x$ (given $x_0$) or am I stuck? Is there another more (mathematically) convenient distribution for $\mu|x_0$ that is typically used in these cases?
A simpler way to look at this, without needing to use the density, is to recognise that $\mathcal{N}(\mu,\sigma^2)\sim\mathcal{N}(0,\sigma^2)+\mu$. From that, you can say
\begin{align} x\vert x_0 &\sim \mathcal{N}(0,\sigma^2)+\mu\vert x_0\\ &\sim \mathcal{N}(0,\sigma^2)+\mathcal{N}(0,\sigma^2)+x_0\\ &\sim\mathcal{N}(0,2\sigma^2)+x_0\\ &\sim\mathcal{N}(x_0,2\sigma^2) \end{align}