I came across this equation while reading introduction to machine learning by Ehtem Alpaydin.
I don't understand how can equation (4.18) be derived using the other two . I have tried using the baye's formula but the normalizing factor is not given. So how can it be derived ?
If you are happy enough with the standard application of Bayes Theorem then the important thing to realise is that we can ignore the normalising constant, carry out a sequence of manipulations on the unnormalised density and then try to normalise at the end by either explicit integration or by noticing similarities with recognised distributions. Finally it also often helps to work with the logarithm and so we have $$ p(\theta | X ) \propto p(X | \theta) p(\theta) \implies \ln p(\theta | X) \propto \ln p(X|\theta) + \ln p(\theta). $$ Carrying out these steps for this example we have \begin{align} \ln p(\theta | X) &\propto -\frac{\sum_i (x_i - \theta)^2}{2\sigma^2}-\frac{(\theta - \mu_0)^2}{2\sigma_0^2} \\ &= - \left\{ \frac{N\theta^2 - N\bar{x}\theta}{2\sigma^2} + \frac{\theta^2 - 2\theta \mu_0 + \mu_0^2}{2\sigma_0^2}\right\} - \frac{\sum_i x_i^2}{2\sigma^2} \end{align} Now since this is a distribution in $\theta$ we are interested only in the quadratic term inside the curly brackets, indeed we have the quadratic \begin{align} Q(\theta) &= a\theta^2 + b \theta + c \\ &= \left( \frac{N}{2\sigma^2} + \frac{1}{2\sigma_0^2}\right)\theta^2 + \left(-\frac{N}{\sigma^2}\bar{x} -\frac{\mu_0}{\sigma_0^2} \right)\theta + \frac{\mu_0^2}{2\sigma_0^2} \end{align} Completing the square we have $$ Q(\theta) = a(\theta - d)^2 + e $$ where $d = b/2a$ and therefore we have that $\ln p(\theta | X)$ is a quadratic in $\theta$ which we recognise as being a Normal distribution and the particular value of the mean is given by $d$ in the quadratic form which on inspection is given by $$ \mu_{\theta | X}= \frac{ N/ \sigma^2}{N/ \sigma^2 + 1/\sigma_0^2}\bar{x} + \frac{1 /\sigma_0^2}{N / \sigma^2 + 1/\sigma_0^2 }\mu_0. $$