Why is there an inconsistency between manual calculations result and R calculation result in t-test?

Question

Initially, I asked this question in "Cross Validated" but since no clear answer was received I moved the question to here.

I have the following sample:

tree length : 186, 181, 176, 149, 184, 190, 158, 139, 175, 148, 152, 111

after using some nutrient:

tree length : 181, 191, 186, 129, 178, 194, 139, 122, 195, 158, 158, 104

I am using Welch two sample t-test with two-tailed testing to see if the nutrient had an effect on tree growth therefore,

$H_0$: Nutrient had no effect.

$H_1$: Nutrient had an effect.

The results in R are as follows:

l1 = c(186, 181, 176, 149, 184, 190, 158, 139, 175, 148, 152, 111)
summary(l1); length(l1);  sd(l1)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  111.0   148.8   166.5   162.4   181.8   190.0 
[1] 12
[1] 23.70063

l2 = c(181, 191, 186, 129, 178, 194, 139, 122, 195, 158, 158, 104)
summary(l2); length(l2);  sd(l2)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  104.0   136.5   168.0   161.2   187.2   195.0 
[1] 12
[1] 31.26754

t.test(l1, l2, alternative = c("two.sided"), paired = FALSE, var.equal = FALSE, conf.level = 0.95)

    Welch Two Sample t-test

data:  l1 and l2
t = 0.10301, df = 20.503, p-value = 0.919
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -22.42215  24.75548
sample estimates:
mean of x mean of y 
 162.4167  161.2500 

Based on the results, $H_0$ is rejected as there has been an effect.

Now, I do the calculations manually:

$t_\alpha = \frac{\mu_1 - \mu_2}{\sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}} = \frac{162.4 - 161.2}{\sqrt{ \frac{23.7^2}{12} + \frac{31.268^2}{12}}} \approx 0.11$

I get the t-value to be 0.11 where there is a 0.01 difference between this result and the R's one (probably due to the different degree of freedom? R uses 20.503 whereas I am using 22 in my calculations). Then based on the significance level of 0.05 and degree of freedom 22, I find the critical value to be 2.074 in the t-distribution table. So since 0.11 < 2.074 $\Rightarrow t_\alpha$ < critical-value, the null hypothesis is failed to reject. But this is inconsistence with R result as in R the null hypothesis was rejected and alternative hypothesis was supported: alternative hypothesis: true difference in means is not equal to 0. So why is there an inconsistency between R result and manual calculation result?

Answer 1

There are at least three issues:

• Neither your manual calculation nor the R automated calculation reject the null hypothesis, and you can see this from the high $p$-value and the confidence interval overlapping $0$. The critical value from $R$ would be qt(0.975,20.503) i.e. about $2.083$, much larger than its $t \approx 0.103$ and not far from your critical value of $2.074$
• rounding of the means: $162.4-161.2=1.2$ but $162.4167 - 161.2500 = 1.1667$; this can make the $t$-value about $0.106$ (rounding to $0.11$) rather than $0.103$
• the number of degrees of freedom: when Welch's $t$-test is used when the two variances are not assumed to be equal, there is the complicated expression below for approximating the degrees of freedom (usually not an integer) so your $\nu=(12-1)+(12-1) =22$ would be based on an assumption the variances are equal while removing this assumptions gives about $20.5$. $$\nu \quad \approx \quad {\dfrac{\left( \; {s_1^2 \over n_1} \; + \; {s_2^2 \over n_2} \; \right)^2 } { \quad {s_1^4 \over n_1^2 (n_1-1)} \; + \; {s_2^4 \over n_2^2 (n_2-1) } \quad }}$$