Chi square goodness-of-fit test for Uniform distribution using Matlab

Question

I need to test random numbers generators in 1, 2 and 3 dimensions. Criteria of test is that generated numbers are from uniform distribution. I am doing this using Matlab. I'm told to use Chi-2 test statistic $$\chi^2=\sum^B_{i=1}\frac{(expected(i)-observed(i))^2}{expected(i)}$$ where $B$ is number of bins, $expected(i)=\frac{N}{B}$ for all $i$ is expected number of elements in $i_{th}$ bin and $N$ is number of generated numbers.

What I have done is that I generated

• $N\times 1$ matrix of random numbers for $1D$
• $N\times 2$ matrix of random numbers for $2D$
• $N\times 3$ matrix of random numbers for $3D$

Then, I set a number of bins (B) for each component of matrix elements and I created zero matrices of sizes

• $1\times B$ for $1D$
• $B\times B$ for $2D$
• $B\times B\times B$ for $3D$

and, using loops, I counted how many generated numbers are in which bin and those were, actually, values $observed(i)$. At this point I am able to calculate $\chi^2$ test statistic for all 3 dimensions.

I know how to read $\chi^2$ table but I don't know how to check significance using Matlab. I'm also not sure about degrees of freedom, but I think it's $B-1$ for $1D$, $BB-1$ for $2D$ and $BBB-1$ for $3D$ since we don't have to estimate any parameter for uniform distribution. I would appreciate some help. Thanks!

Answer 1

That all sounds right; looks like you're almost done. All you need to do now is call chi2cdf(x,v,'upper') where $x$ is your $\chi^2$ test statistic and $v$ is your degrees of freedom; this will output the P-value for the test.

Answer 2

Just to expend on it and give the all the code.

Setting parameters

N=100; % sample size
a=0; % lower boundary
b=1; % higher boundary

Sample N uniformly distributed values between a and b. And in the second line add some bais to make it not uniform if you want to test the code.

x=unifrnd(a,b,N,1);
%x(x<.9) = rand(sum(x<.9),1);

Using chi2gof

As described here, with chi2gof, you can't use the 'cdf of the hypothesized distribution' and need to specified the bins, the edges and the expected values.

nbins = 10; % number of bin
edges = linspace(a,b,nbins+1); % edges of the bins
E = N/nbins*ones(nbins,1); % expected value (equal for uniform dist)

[h,p,stats] = chi2gof(x,'Expected',E,'Edges',edges)

Using chi2cdf

With this function you need to supply the chi-squared test statistic, $\displaystyle \chi ^{2}$ which can be computed with the function histogramm:

h = histogram(x,edges);
chi = sum((h.Values - N/nbins).^2 / (N/nbins));
k = nbins-1; % degree of freedom
chi2cdf(chi, k)

Note, that if you don't use the edges to compute the number of value per bins, histogramm will choose them from the lower value to the highest and therefore the final score will be different than with chi2gof

Quick Theory recall

Just to recall how to interpret the final value, few definition:

• The null hypothesis ($H_0$) is that the data x are coming from a uniform distribution.
• Pearson's chi-squared test is testing if you can safely reject the null hypothesis, i.e. "Can I say that x is not a coming from a uniform distribution ? "
• Pearson's cumulative test statistic $\displaystyle \chi ^{2}$ is a measure of the error between observations and expected value $$\displaystyle \chi ^{2} = \sum_{i=1}^N \frac{(O_i-E_i)^2}{E_i}$$
• Chi-squared distribution is the distribution that the Pearson's cumulative test statistic $\displaystyle \chi ^{2}$ would follow according $H_0$ (i.e. if the observation are coming from a uniform distribution)
• The p-value is the probability of obtaining a worst result than what was observation, when the null hypothesis is true. That is, the probability that, if we randomly draw a dataset y in a uniform distribution, the error (or Pearson's cumulative test) will be equal of higher than the actual observation x (=worst case).
• So, we can reject $H_0$ if p is lower than a significant level $\alpha$. That is, for small value of p, we can safely say that x is not coming from a uniform distribution.