When lower limit of integration is a function of the variable of differentiation while the upper limit is positive infinity, in this case, how do I deal with the upper limit which is not a function? I understand that if it's a constant, then we can safely ignore it, but when it's infinity, can I still do that?
Thanks in advance!
Yes, you can (provided that the integral converges). The reason is that $$ \int_{\alpha (t)}^\infty f(x)\,dx =\int_{\alpha (t)}^M f(x)\,dx + \int_M^\infty f(x)\,dx $$ by the additivity of integral over domain. Taking derivative of the right hand side with respect to $t$, you are dealing with a proper integral, plus a constant (independent of $t$).
The number $M$ can be any constant such that $\int_M^\infty f(x)\,dx$ is defined and converges; e.g., for the function $f(x)=1/x^2$ any $M>0$ will do. But it's somewhat natural to choose $M$ so that it's larger than the values that $\alpha$ takes on; this way, the integral $\int_{\alpha (t)}^M f(x)\,dx$ has limits placed in the natural order.