Say we have the symmetric group, $S_n$, and we take a subgroup, $H$, specified by the rule that $\sigma(5) = 5$. The question is: what is the cardinality of $H$?
After working out some examples, it seems logical that the answer is $(n-1)!$. I believe the argument is that we're fixing a single element, $5$, by mapping it to itself, but are permuting each of the remaining elements, i.e., we're permutting $n-1$ elements. Similarly, we could say that there are $n!$ total permutations in $S_n$, and we want to discard all those permutations that send $\sigma(5)$ to something else. So, there are $n-1$ possibilities for $\sigma(5)$ and $(n-2)!$ combinations for all of the remaining elements, as we've already fixed some value to send $5$ to, so the result is $(n-1)(n-2)! = (n-1)!$.
How does this logic sound?
This sounds exactly right. In fact, if for any given $x,y\in\{1,...,n\}$ the subset $H=\{\sigma\in S_n:\sigma(x)=y\}$ will have cardinality $(n-1)!$ using pretty much the same argument you mentioned above. In the case $x=y$, $H$ will be a subgroup of $S_n$ isomorphic to $S_{n-1}$.