$\chi^2$ test for comparison of distributions

47 Views Asked by At

I have some measurements that should follow a Poissonian distribution, I have to check whether or not the hypothesis that the data follow the distribution is statistically acceptable, to do this I have to use the $\chi^2$ test. I'm trying to wrap my head around it but I just can't find good resources on the internet so I'll ask here.

If I understood well, I should calculate the measurments $\chi^2$ value

$$ \chi^2_m=\sum_{n=0}^k\frac{(x_n-Np_n)^2}{Np_n}$$

where $k$ is the number of bins in which I divide my distribution $x_n$ is the number of points in the $n^{th}$ bin, $p_n$ is the value of the Poissonian pdf at the center of the $n^{th}$ bin and $N$ the total number of points measured. To check whether or not the value of $\chi^2_m$ is acceptable, I have two options:

-I can look up the critical value of $\chi^2$ with $k-1$ DoF (because of the constraint $\sum x_n=N$) on some table for some p-value, say 0.05, and if my $\chi^2_m$ is smaller than the one on the table I cannot reject the null hypothesis, i.e. I accept that the distribution is indeed Poissonian. The problem I see with this is that obviously the smaller the p-value is, the greater $\chi^2$ is, thus the choice of the p-value is irrelevant: I can choose it as small as I want after the a certain value.

-Another possibility is to integrate the $\chi^2$ distribution with $k-1$ DoF from $\chi^2_m$ to infinity: this should output the probability that the true $\chi^2$ is greater than the one of the measurement, i.e. the probability that the null hypothesis is rejected even though it was correct (is this the p-value?). I understand the logic of this better but when I do the calculations I obtain absurd results.

My questions are:

1) Where is the fault in my reasoning for the first method?

2) Is the second method correct?

3) Why do we only care if the $\chi^2$ that we measure is greater than a certain value? For many DoF the $\chi^2$ distribution is similar to a normal distribution, so it has a tail on the left too.