Finding Posterior Distribution for Simple Linear Regression

36 Views Asked by At

A simple linear model is given as $$ Z_i = \gamma_1 + \gamma_2 y_i + \epsilon_i$$ $i=1, \ldots,n.$ Let $\mu = (\gamma_1, \gamma_2)'$. Assuming that $\epsilon_i \sim N(0,1)$ and using a noninformative prior for $\mu,$ find the posterior distribution of $\mu.$

Below are my work:

In matrix form $ y = Y \mu + e$, $ e \sim N( 0, \sigma^2 I).$ Using a constant prior: $p(\mu) \propto c.$

$$p(\mu \mid y) = p( y \mid \mu) \cdot \tau ( \mu)=(2\pi \sigma^2 )^{-n/2} \exp ( -\frac{1}{2} \sigma^2 ( y - Y \mu)' ( y - Y \mu) ) \cdot c\propto \exp ( -\frac{1}{2} \sigma^2 ( y - Y \mu)' ( y - Y \mu) )$$

where after some algebra

$$( y - Y \mu)' ( y - Y \mu) =( y - Y \widehat{\mu})' ( y - Y \widehat{ \mu)} + 2 (\widehat{ \mu} - \mu)' Y' ( y - Y \widehat{ \mu})+( \mu - \widehat{ \mu})' Y'Y ( \mu - \widehat{ \mu}),$$

Using the Normal equation, $Y'Y\widehat{ \mu} = Y' y$ the middle term is zero(0) so that

$$( y - Y \mu)' ( y - Y \mu) =( y - Y \widehat{\mu})' ( y - Y \widehat{ \mu)} +( \mu - \widehat{ \mu})' Y'Y ( \mu - \widehat{ \mu})$$

Therefore, $$p(\mu \mid y) \propto \exp ( -\frac{1}{2} \sigma^2 ( y - Y \mu)' ( y - Y \mu) ) \propto \exp ( -\frac{1}{2} \sigma^2 ( y - Y \widehat{\mu})' ( y - Y \widehat{ \mu)} +( \mu - \widehat{ \mu})' Y'Y ( \mu - \widehat{ \mu}) ).$$

and

$$p(\mu \mid y) \propto \exp ( -\frac{( \mu - \widehat{ \mu})' Y'Y ( \mu - \widehat{ \mu})}{2\sigma^2} ) \sim N ( \widehat{ \mu}, \ \sigma^2 (Y'Y)^{-1} ).$$

ANALYZE MY WORK. THIS IS A SELF STUDY PROBLEM.