I have a two-dimensional random vector $\mathbf{x} = [x_1, x_2]^T$ with a known joint probability density function (pdf). The pdf is non-Gaussian and the two entries of the random vector are statistically dependent. I need to show that for example $x_1$ is more important than $x_2$, in terms of the amount of information that it carries. Is there a classical solution for this problem? Can I show that for example n% of the total information carried by $\mathbf{x}$ is in $x_1$ and 100-n% is carried by $x_2$?
I assume that the standard way of measuring the amount of information is by calculating the Entropy. Any clues?
The standard way to "measure the (average) information" of a random variable is just to compute its entropy. By that reasoning, you would say that $X_1$ gives more information than $X_2$ if $H(X_1)>H(X_2)$. Note that, for this, the fact that the variables are or not independent, is irrelevant.
However, that makes sense only for discrete variables. For continuous variables, the entropy is infinite - and the "differential entropy", though generally finite, is not a very useful measure of information content - for one thing, it depends on the scale.
The "total (joint) information" is $H(X_1,X_2) = H(X_1)+H(X_2)-I(X_1;X_2)$ Hence it's hard to make sense of such a division.