Can we decompose the likelihood function of a composite model into the likelihoods of of its parts?

56 Views Asked by At

Let's say we have some models $f,f'$ with parameters $\theta_f$ and $\theta_f'$. Can we generally express the conditional likelihood function of $f' \circ f$ in terms of the likelihoods of $f$ and $f'$? That is, is there some function $g$ such that we can write the following? \begin{align*} L_{f' \circ f} = g(L_{f}, L_{f'}) \end{align*} Perhaps there is some measure $\mu$ for which we can write something like the following for a single data sample? \begin{align*} L_{f' \circ f}(a,c, \theta_1 \oplus \theta_2) = \int_b L_f(a,b, \theta_1)L_{f'}(b,c,\theta_2) d\mu \end{align*} For example, $f$ and $f'$ might be linear regression models, in which case: \begin{align*} f(a) = \beta_1 a + \beta_0\\ f'(b) = \beta_1' b + \beta_0'\\ (f' \circ f)(a) = \beta_1'(\beta_1 a + \beta_0) + \beta_0' \end{align*} In this case I believe that the likelihood functions of these models for a single data sample would be: \begin{align*} L_{f}(a,b) = \mathcal{N}(b - \beta_1 a - \beta_0, \sigma)\\ L_{f'}(b,c) = \mathcal{N}(c - \beta_1' b - \beta_0', \sigma')\\ L_{f' \circ f}(a,c) = \mathcal{N}(c - \beta_1'\beta_1 a + \beta_1' \beta_0 + \beta_0', \beta_0'\sigma + \sigma') \end{align*}

1

There are 1 best solutions below

0
On BEST ANSWER

Okay, I think I have figured this out. I believe our goal is to express the conditional likelihood function: \begin{align*} P(c | a,\theta_1,\theta_2) = L_{f'\circ f}(a,c,\theta_1 \oplus \theta_2) \end{align*} In terms of: \begin{align*} P(b | a,\theta_1) = L_{f}(a,b, \theta_1)\\ P(c | b,\theta_2) = L_{f'}(b,c, \theta_2) \end{align*} We can see that: \begin{align*} P(c | a,b,\theta_1,\theta_2) = P(c | b,\theta_2)\\ P(b | a,\theta_1,\theta_2) = P(b | a,\theta_1) \end{align*} So by Bayes rule we can write: \begin{align*} \frac{P(b,c | a,\theta_1,\theta_2)}{P(b | a,\theta_1,\theta_2)} = P(c | b,a,\theta_1,\theta_2)\\ P(b,c | a,\theta_1,\theta_2) = P(c | b,a,\theta_1,\theta_2)P(b | a,\theta_1,\theta_2)\\ P(b,c | a,\theta_1,\theta_2) = P(c | b,\theta_2)P(b | a,\theta_1)\\ \int_b P(b,c | a,\theta_1,\theta_2) db = \int_b P(c | b,\theta_2)P(b | a,\theta_1) db \\ P(c | a,\theta_1,\theta_2) = \int_b P(c | b,\theta_2)P(b | a,\theta_1) db \end{align*}