I have 2 events:
Event1 Event2
a 1
a 1
a 2
a 2
b 1
b 1
b 1
b 1
b 2
I want to calculate the mutual information between these 2 events. Here how I calculate it.
First I get the probability: P(a) = 4/9 , P(b) = 5/9 , P(1) = 6/9 , P(2) = 3/9
Then the joint probability:
P(a,1) = 4/9 x 6/9 = 24/81
P(a,2) = 4/9 x 3/9 = 12/81
P(b,1) = 5/9 x 6/9 = 30/81
P(b,2) = 5/9 x 3/9 = 15/81
After that, I use the formula of Mutual Information:
MI(Event1, Event2) = 24/81 log (24/81 / 4/9 x 6/9) + 12/81 log (12/81 / 4/9 x 3/9) x 30/81 log (30/81 / 5/9 x 6/9) + 15/81 log (15/81 / 5/9 x 3/9)
Is this correct?
Not really.
In particular, it seems that you are assuming $$P(a,1)=P(a)P(1)$$ which is not true.
Try to count how many $(a,1)$ occur to get an estimate of $P(a,1)$.
Remark: if we assume that $P(x,y)=P(x)P(y)$, the mutual information is equal to $0$.