I have started learning the very basic distribution theory and I encountered the idea of a test function and distribution. I am not entirely sure why the following definition of a test function is necessary.
A test function is a $C^{\infty}$ function such that it admits a compact support.
I was wondering why do we need it to be in $C^{\infty}?$ Is it because if $f$ is a generalised function we would want $f^{(n)}$ to also be a generalised for any $n$? But even if that is true, what is the benefit of this?
I would say that it also allows to have a largest space. Since the smaller is the space of test functions, the bigger is the dual space. For example bounded measures can be seen as the dual of continuous functions, and so there are less measures than distributions. But yes, it also allows you to differentiate as many times as you want ... this is often useful in applications, such as partial differential equations. If you prefer, knowing that the set of distribution is stable under differentiation tells you that distributions can be as irregular as you want.
Let take for example the Dirac delta. You can see the Dirac delta as a functional over continuous functions: $\delta_0\in (C^0)'$ defined for every $\varphi\in C^0$ by $$ \langle \delta_0,\varphi\rangle = \varphi(0) $$ If you do not it to take derivatives of it, this approach is sufficient. If you need to take its first derivative you can look at the same definition with $\varphi\in C^1$, and then you can define its derivative $\delta_0'$. One of the goals of distributions is to have one big space to put all distributions in order not to have to care about the precise space. Since the space is bigger, you will have more objects available in it and so it will be easier to prove existence theorems for PDE for example. However, in exchange, you lose the knowledge about the regularity of your solution.
Then, to study regularity, one often uses Sobolev spaces $W^{s,p}$ (or more refined scales). The set distribution of order $n$ (so the distributions for which you just need $\varphi\in C^n$) contain the spaces $W^{-n,p}$.
This is also useful to generalize the Fourier transform. The Fourier transform of a function as simple as $\mathbf{1}_{\mathbb{R}_+}$ is a distribution of order $1$, and in general it is not simple to know in what space exactly will be the Fourier transform of of a function that is not in $L^p$ with $p\leq 2$, so it is good not to have to worry about the regularity in this case.