Degrees of Freedom for Student's t-Distribution

Question

The student's T statistic is given by the formula - t = [X(bar) - Mu]/ [S/sqrt(n)]

X(bar) is sample mean, Mu is population mean, S is the mean square

I have interpreted the degrees of freedom in the following way, please let me know if I am making a mistake.

In all the exact sampling distributions, there's one assumption that's always constant i.e. parent population is always normal. Going by this assumption, the statistic [X(bar) - Mu]/[Sigma/sqrt(n)] would follow a normal distribution without the aid of Central limit theorem that necessitates the size of sample to be large. The degrees of freedom of this statistic would be n since n values can vary independently. But for small samples like this one, a possible estimator of Population variance Sigma^2 can be the mean square, S^2. If we estimate sigma by S , we'll have to reduce our degrees of freedom by 1 and hence the degrees of freedom for the statistic -

t = [X(bar) - Mu]/[S/sqrt(n)] becomes (n-1)

Is this interpretation of Student's t statistic correct?

Answer 1

Theoretically, the degrees of freedom parameter $\nu = n-1$ is due to the definition of Student's t distribution. You are correct that this distribution converges standard normal for increasing $n.$

Intuitively, people sometimes think of DF in terms of dimensionality: A sample of size $n$ exists in $n$-dimensional space. One dimension is "used" to find the sample mean $\bar X = \frac 1n\sum_{i=1}^n X_i.$ Then the remaining $n-1$ dimensions are "used" to find the sample variance $S^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i-\bar X)^2.$ The use of $n-1$ in the definition of $S^2$ implies that $E(S^2) = \sigma^2,$ the population variance.

If I take many samples $m$ of size $n=5$ from $\mathsf{Norm}(\mu=100,\sigma=15)$ and then compute t-statistics $T$ for each sample under the null hypothesis that $\mu=100,$ then a histogram of the $m$ values of $T$ will closely match the density function of Student's t distribution with $\nu = n-1 = 4.$

In R, I can find the t-statistic for one sample of size $n=5$ as follows:

set.seed(111)
x = rnorm(5, 100, 15);  x
[1] 103.52831  95.03896  95.32564  65.46482  97.43686
t = t.test(x, mu=100)$stat; t
        t 
-1.299141  

The R code below computes $m$ such t-statistics and puts them into a vactor t.stat of length $m,$ makes a histogram of the $m$ values, and shows the density function of $\mathsf{T}(\nu=4)$ for comparison. [The t-statistic shown above is the first of the $m$ statistics sampled in this way.]

set.seed(111)
m = 10^5;  n = 5;  mu.0 = 100;  sg = 15
t.stat = replicate(m, t.test(rnorm(n,mu.0,sg),mu=mu.0)$stat)
summary(t.stat)
      Min.    1st Qu.     Median       Mean    3rd Qu.       Max. 
-24.220863  -0.741238  -0.001904  -0.000634   0.735395  25.473347 
var(t.stat)
[1] 1.992431   # aprx Var(T) = 2


hist(t.stat, prob=T, br=100, ylim=c(0,.4), col="skyblue2")
 curve(dt(x,n-1), add=T, lwd=2, col="red", n=10001)

Notice that the distribution of $\mathsf{T}(\nu=4)$ has a higher variance $(2)$ than does standard normal $(1).$