Let's say we have : $$ X_i \sim \ N( \mu, \sigma^2 ) $$ iid
I'm constructing this test function, in order to test two hypothesis on $\mu$ : $$ \mathbb { 1} {\{ \sum^n X_i < q_a \} } $$ where $q_a$ is the $a$-quantile of $$ P_{ \mu_0 } ( \sum^n X_i < q_a ) $$
If I know $\sigma$, this test is just fine. Because then I can rewrite the test function as : $$ \mathbb { 1} \{ \frac{ \sum X_i - n \mu_0 }{ \sigma \sqrt{n} } < \phi^{-1} (a) \} $$
But if I don't know $\sigma$, how can I handle things ? I have been told to replace it by $S$, such that : $$ S^2 = \frac 1 {n-1} \sum^n (X_i - \overline X)^2 $$
But how do you conclude that the test is now :
$$ \mathbb { 1} \{ \frac{ \sum X_i - n \mu_0 }{ S \sqrt{n} } < t_{n-1, a} \} $$ where $t_{n-1, a} $ is the $a$-quantile of the student law with n-1 degrees of freedom?
The idea is that we are testing for two different things. The first one is a test for a normal where we know $ \sigma $, the second for an unknown $\sigma$. We aren't replacing anything in the first case to get to the second. The thing is it is two totally different ideas. The only link I found is that :
$ \bar{X} $ is independant to $ S^2$. Hence :
$$ \frac {\bar{X} - \mu } { \frac {S}{ \sqrt n } } = \frac { \frac { \sqrt n (\bar{X} - \mu ) }{ \sigma } }{\sqrt {\frac { (n-1) S^2 }{(n-1) \sigma^2 } } } \sim t_{n-1} $$
In both case we want a test with $\alpha$ level of risk, so take the $1-\alpha$ quantile. Which leads us to the result mentionned in the question.
So the conclusion is, we have two different tests, asssociated to two different random variables, depending on whether we know the variance or not.