Estimating joint probability distributions from marginal distributions

2.4k Views Asked by At

I'm currently studying a problem where I have well determined marginal distributions for N random variables that conform to the beta distribution and very few samples of the joint distribution. I want to leverage the marginal distributions to estimate the joint distribution is this feasible?

For a concrete case, there may be 10k to 100k samples of the marginal distributions and fewer than 100 samples of any given joint distribution.

1

There are 1 best solutions below

0
On BEST ANSWER

I can maybe try to give you a hint where your problem crops up. You an use Sklar's theorem (https://en.wikipedia.org/wiki/Copula_(probability_theory)#Sklar.27s_theorem) to decompose the joint distribution into marginal contributions and the impact of the dependence structure. To be precise denote by $H$ the joint distribution function of the random variables $X_1,\ldots,X_d$, where the marginal distribution are $X_i \sim F_i$. By Sklar's theorem there exists a function $C$ such that \begin{align*} H(x_1,\ldots,x_d) = C(F_1(x_1),\ldots,F_d(x_d)). \end{align*} To estimate $H$ you need to estimate $F_i$, $i=1,\ldots,d$ and $C$. Typically, and this fits well with your framework, people first find estimates $\hat{F}_i$ for the margins, where each estimation is carried out separately.

In a second step the joint observations $\mathbf{x}_i = (x_{i,1},\ldots,x_{i,d})$ are then used to create pseudo observations $\mathbf{u}_i = (\hat{F}_1(x_{i,1}),\ldots,\hat{F}_d(x_{i,d}))$. These are then used to obtain an estimate $\hat{C}$ for $C$.

While usually the sample sizes for the margins and the joint distribution are the same, in your setting you can estimate the margins much more precise than the dependence structure. This means that you will create good pseudo observations, but it does not alleviate you from estimating $C$ based on the lower sample size.

To sum up, your setting actually eliminates (not completely but pretty much) the error that is usually encountered when forming the pseudo observation. The final estimation of $C$ is however still based on the smaller sample size. Depending on the dimension $d$, the selected copula family, the sample size and your estimation method (parametric, semi-parametric, non-parametric) the sample size might be enough - or not...


To get started with copulas there is a good introduction: Roger B. Nelsen (1999), "An Introduction to Copulas", Springer.

Splitting the inference in the two steps outlined above is usually referred to as Inference for Margins (IFM) approach - here google is your friend...