Existence of Taylor-like expansion in perturbation parameter

Question

In perturbation theory, we assume that a Taylor-like series exists in the perturbation parameter. But is this always true? Or are there certain conditions? Also, can I get some resource recommendations where I can read more about the theory of perturbation theory? Theory as in a rigorous explanation of why it works, not the different kinds of perturbative approaches.