Why do I compute diffirent F values in R.

Question

So I have some data set and it contains two factor predictors, $X1$ with $3$ levels and $X2$ with $2$ levels, I want to compare the models

$$1+X1$$ vs $$1+X1+X2+X1:X2 $$ that is , the complex model contain interaction.

First of all, ( I am wondering if this is correct reasoning)

I set for example , fit1<- lm(y~1+X1) , and then used anova(fit.1) to read what came up for Sum Sq column and residual row. So would that be SSres of my simple model?

I did the same for the complex model to get what I thought was SSres(complex)

Since the complex model has 6 parameters, the data contains 36 measurements, and the simple model contains 3 parameters, I computed

$$F=\frac{SSres(simple)-SSres(complex)/r}{SSres(complex)/n-p}$$

where $n-p=30$ and $r=3$

This gives me an answer of $F=9.48$

However,

doing fitcomplex<-lm($y~1+X1+X2+X1:X2$)

and then doing drop1(fitcomplex,test='F')

gives me an $F=6.147$

So why do I get different answers for this? Which is the correct one? What is the relation between them?

Thanks for any help

Answer 1

Comments:

Without seeing the data and the ANOVA tables, it is difficult for me to give a sharply targeted answer. So here are some comments that might help.

1) Make sure all factor variables are declared as type factor. Otherwise, they will be used as numerical variables (covariates or explanatory variables for regression).

2) It is not a good idea to compare F-ratios in this way. F-distributions with different degrees of freedom have different shapes. It makes more sense to compare P-values or R-squared values. If 'R-sq' (not 'R-sq-adj') is smaller for the more complex model, that would be surprising; you ought to get a 'better fit' to the more complex model.

3) If you don't get an answer on this site that is useful, you might want to try our sister site 'cross-validated' which is often more attuned to issues in applied statistics. But if you do so, try to give them more details to work with than you have given here.