What is the simplified formula to calculate joint conditional entropy of 4 or more variables for instance H(a|b,c,d) or H(a|b,c,d,e)?

146 Views Asked by At

I'm a medical science student and I came across a point in my research I've to work with entropy. As entropy involve probability theory which , like many, I'm not good at ;)

I can calculate an H(a) or entropy for any other variable using an online calculator.

I know that H(a|b,c) = H(a,b,c) - H (b) - H(c|b).

However, now I have to calculate H(a|b,c,d) and H(a|b,c,d,e). It'll be really nice if someone expert in math and probabilities can answer how to expand the formula to calculate H(a|b,c,d) and H(a|b,c,d,e).

And if someone can point me to an online calculator which can calculate all these entropies than it'll ease out a lot of work for me as I've to calculate it for several combinations of variables.

I'll be thankful for the help :)

1

There are 1 best solutions below

2
On BEST ANSWER

Hint: use the chain rule $$H(X_1,\cdots ,X_n)=H(X_1)+H(X_2|X_1)+H(X_3|X_2,X_1)+\cdots+H(X_n|X_{n-1},\cdots ,X_1)$$