Causality in Vaccine versus Placebo Group

Question

I am having a debate with some people on statistical significance of a certain event.

It is based on a real world situation, namely the AstraZeneca vaccine and the incidence of Transverse Myelitis.

As far as I know the trial invovled two groups with about 25,000 people in each group. (50,000 people altogether).

In the vaccine group there were two incidences of Transverse Myelitis, and in the placebo group there was one incidence.

Without going into the maths I can clearly see that there is no statistical significance of Transverse Myelitis in these two groups. The numbers are just too small.

Can anyone give me a basic mathematical breakdown of this situation? I would just like to back up my suspicions with some cold hard maths.

Thank you.

Answer 1

Suppose you "randomly" selected from the general population until you got exactly 3 people with TM. Now (randomly) split the group into 2 equal groups. What is the probability that the sick people will be split 1:2? Since there is 0.5 probability for each TM patient to be in either of the groups, the probability to see exactly 2 in a specific group is 3/8. That is not too bad. If you ask what is the probability to have 0:3 or 1:2, the odds increase to exactly 1/2 ( just from symmetry).

Answer 2

Since we don't know what's the chance of catching TM is, we will use the placebo group as reference. That is, the chance of catching it (within the duration of the study) is 1 in 25k.

Now imagine you choose from blind 25k people from a population where 1 in 25k has TM. Then the possibility of picking up 2 or more TM cases is approximately $1-2e^{-1}\approx 26\%$ (the exact number is $1-(\frac{24999}{25000})^{25000}-(\frac{24999}{25000})^{24999}=0.264241\dots$). Note this is not saying 26% chance of the vaccine does not cause TM or 74% chance of taking the vaccine will cause TM (analogy: the chance of getting a six on a fair die is 1/6, the fact that you rolled once and get a six doesn't mean the probability of this six happening not from randomness but from magic is 5/6).

That is, not counting publication bias. If the vaccine group also had 1 case then there was nothing for the antivaxxer to winne about, and if there are none it also wouldn't make the news (nobody sane would use it to claim the vaccine also vaccinate against TM).