Bayesian posterior with unknown parameters of normal distribution

64 Views Asked by At

I was wondering if I could get help with a Bayesian inference problem involving a normal distribution signal. In this problem, the world can be in one of two states: high (H) or low (L). If the world is in state H, a parameter $\theta$ is drawn from a normal distribution $N(\mu_H,\sigma_H^2)$. If it's in state L, it is drawn from $N(\mu_L,\sigma_L^2)$. However, the actual value of $\theta$ is unknown.

The world observes $y_{t}$ at each period $t$ where $y_{t}= \theta + \epsilon_{t}$ and $\epsilon_{t}$ ~ $N(0,\sigma_{\epsilon}^2)$. This is a standard Bayesian inference problem with a normal distribution signal.

Suppose the world has a prior $q_{0}$ that the state is high. After observing the data $D_{t} \equiv \{y_{1},y_{2},...,y_{t}\}$, the posterior $q_{t}$ is given by:

$$q_{t} = \frac{q_{0} \int P(\theta|H) \cdot P(Data_{t}|\theta,H) d\theta}{q_{0}\int P(\theta|H) \cdot P(Data_{t}|\theta,H) d\theta+ (1-q_{0})\int P(\theta|L) \cdot P(Data_{t}|\theta,L) d\theta} $$

The probability density functions are given by:

$$P(\theta|H) = \frac{1}{\sqrt{2\pi\sigma_{H}^2}}e ^{-\frac{1}{2}(\frac{\theta-\mu_{H}}{\sigma_{H}})^2} $$

and

$$P(Data_{t}|\theta,H) = \prod_{i=1}^{t} \frac{1}{\sqrt{2\pi\sigma_{\epsilon}^2}}e ^{-\frac{1}{2}(\frac{y_{i}-\theta}{\sigma_{\epsilon}})^2} $$

A posterior $q_{t}$ is a function of $Data_{t}$.

My questions are (1) whether it is possible to express $q_t$ as a function of the sum of the data $\sum_{i=1}^{t} y_i$ instead of the vector ${y_1, y_2, ..., y_t}$ and (2) whether I can recursively express $q_{t}$ by $q_{t-1}$ and only with the latest observation ${y_{t}}$.

If any one of them is possible, it will allow me to keep track of $q_t$, not with a full vector of observations. Or, would there be another way to reduce the dimension?

1

There are 1 best solutions below

0
On

In the end, I read about the Gaussian integral. Once I applied the generalized Gaussian integral formula, the Bayesian updating rule became sufficiently simple. The state space became time $t$, the prior $q_{0}$, and $\sum^{t}_{i=1}y_{i}$.