Why does the reduced Chi-squared around 1 not zero?

Question

It is mentioned here that the reduced Chi-squared $\chi^2/dof$ is always compared to 1 and it determines the goodness of fitting, where $\chi^2$ is defined as: $$\chi^2=\sum{\frac{(model-data)^2}{\sigma^2}}$$ Meanwhile as I know the $\chi^2$ represents the sum of the squared residuals, so if the model represents the best fit, does not this lead to the sum of the residuals tends to zero and the best fitting should be zero?
I know that $\sigma$ can be used for scaling, but does it mean that even if we have one set of data and we perform fitting with several models should we use different $\sigma$ for each model. I think that I might have a misconception.

Answer 1

I think you confuse “true model” and “no standard deviation.” We have an assumption that the observations come from a Gaussian generative process $y_i\sim N(X_i^T\beta, \sigma_i^2)$ for each $i$. If this true model is found, the $\chi^2$ statistic $\sum_i \left(\frac{(y_i-f(x_n|\beta))^2}{\sigma_i^2}\right)$ will be $\chi$-square with n df because it is the sum of n Gaussian random variates squared. The expectation is n, so divided by the degrees of freedom, n, we get 1 for the correct model.

If the statistic is greater than 1, generally it implies underfitting (residuals too large), while less than 1 means overfitting. This is done logically by comparison with the fact that it is 1 under the true model, but in practice it can be complicated.

On the other hand, if the standard deviations were all $0$, we’d get a perfect fit with no residuals, but we cannot have a zero standard deviation in the Gaussian model. The model would be deterministic with no randomness. A chi-square statistic would be inappropriate.