Consider $r_t = \mathbb{E}[Z_t]^T\beta$; $t= 1, 2, 3, \cdots$, where $r_t \in \mathbb{R}$, $\beta \in \mathbb{R}^d$ and $Z_t$ is a $d$-dimensional random vector.
Let's say $\beta$ is an unknown parameter and we would like to estimate it with a Bayesian approach by observing $r_t, Z_t$ for $t=1, 2, 3, \cdots$ (There is no iid assumption on random variables $Z_t$ and their distribution is unknown).How should we proceed? Any ideas?
My Approach: In the case that $d=1$, we may say $\mathbb{E}[Z_t] = \frac{1}{\beta}r_t$, and then say that assuming $\frac{1}{\beta} \sim f_\theta$ as a prior and $Z_t = \frac{1}{\beta}r_t$ we get a prior on $Z_t$. Then, we can update $\theta$ with posterior based on our observation $Z_t = z_t$. I have two questions here:
- First, is my understanding correct in the case that $d=1$?
- How can I extend it to $d > 1$?
These are just some ideas for a way to approach this.
Assuming that $r_t = E[Z_t]^T\beta = \beta^TE[Z_t]$, we can place $d(\mu_t, \sigma I_p)$ priors with some location and scale parameter on $Z_t$. If we place $N(\mu_t, \sigma I_p)$ priors we get some analytical tractability.
That is if $Z_t \sim \textrm{N}(\mu_t,\sigma^2 I_p)$, then $r_t \sim \textrm{N}(\beta^T\mu_t,\sigma^2\beta^T\beta)$, and we can place a prior on $\mu_t$, $\beta$, and $\sigma$. It's easier to assume that you at least know to what degree you're uncertain about your estimates $Z_t$ of $E[Z_t]$; meaning that $\sigma^2$ is a parameter to be specified.
In that setting, if $\mu_t \overset{iid}{\sim}\textrm{N}(0,\tau^2 I)$, where $\tau$ is a parameter to give a dispersed uninformative prior, and $\beta \sim \textrm{N}(0,\phi^2 I)$.
The likelihood for the model is then:
\begin{equation} \mathcal{L}(r_t | \mu_t, \beta) = \frac{1}{\sqrt{2\pi\sigma^2\beta^T\beta}}\exp\{-\frac{1}{\sigma^2\beta^T\beta}(r_t - \beta^T\mu_t)\} \end{equation}
The full posterior is then proportional to:
\begin{equation} \pi(\mu_t, \beta | r_t) \propto \frac{1}{\sqrt{2\pi\sigma^2\tau^2\phi^2 \beta^T\beta}}\exp\{-\frac{1}{\sigma^2\beta^T\beta}\sum_{t}(r_t - \beta^T\mu_t)^2\}\exp\{-\frac{1}{\tau^2}\beta^T\beta\}\prod_{t}(\exp\{-\frac{1}{\phi^2}\mu_t^T\mu_t\}) \end{equation}
Which seems like a pretty monstrous posterior to analyze, but should be fairly straightforward using Metropolis Hastings.