Let $P_t$ be a Feller semigroup, i.e. a contractive, positive preserving, sub-Markovian, strongly continuous semigroup with the Feller property on $C_\infty(\mathbb{R}^d)$.
Then I am trying to show that for $\alpha, \epsilon>0$ and $u \in C_\infty(\mathbb{R}^d)$, $\frac{1}{\epsilon} \int_0^\epsilon e^{-\alpha s}P_s uds \to u $ as $\epsilon \to 0$.
I can prove this without the exponential as follows: for $r \ge 0$, $$\bigg|\frac{1}{\epsilon} \int_r^{r+\epsilon} P_su(x)ds-P_ru(x)\bigg|\le \frac{1}{\epsilon}\int_r^{r+\epsilon}|P_s u(x)-P_r u(x)|ds\le \sup_{r \le s \le r+\epsilon} \Vert P_s u -P_r u\Vert_\infty \le \sup_{s \le \epsilon} \Vert P_s u - u \Vert_\infty \to_{\epsilon \to 0} 0,$$ where we use the contraction by $P_{r}$ in the third inequality and the strong continuity at the end.
However, I am having trouble adapting this for $e^{-\alpha s}P_s$. The proof with $r=0$ gives $\sup_{s \le \epsilon}\Vert e^{-\alpha s} P_s u - u \Vert_\infty$, but I cannot use strong continuity here. How can i solve this issue? I would greatly appreciate any help.
To begin, for any strongly continuous semigroup $(T(t))_{t \geq 0}$ on any Banach space $X$, for any $\delta > 0$ one has $$ \bigg\Vert \frac 1 \varepsilon \int_0^\varepsilon T(s) x \, \mathrm d s - x\bigg\Vert = \bigg\Vert \frac 1 \varepsilon \int_0^\varepsilon T(s) x - x \, \mathrm d s \bigg\Vert \leq \frac 1 \varepsilon \int_0^\varepsilon \Vert T(s) x - x \Vert \, \mathrm d s \leq \delta,$$ since one can choose $\varepsilon > 0$ such that $\Vert T(s) x - x \Vert < \delta$ for all $0 \leq s \leq \varepsilon$ by strong continuity. Hence, $\frac 1 \varepsilon \int_0^\varepsilon T(s) x \, \mathrm d s \to x$ as $\varepsilon \to 0$.
So your problem just boils down to observe that the semigroup $(\mathrm e^{-a s} P_s)_{s \geq 0}$ is strongly continuous.
Now it is well-known in semigroup theory that strongly continuous semigroups correspond to abstract Cauchy problems. In particular, if $A$ is the infinitesimal generator of a semigroup $(T(t))_{t \geq 0}$ on some Banach space $X$, then $u(t) := T(t) x_0$ is the unique classical solution of $$ u'(t) = Au(t), \quad u(0) = x_0 \in X.$$ In this case, cf. the great monography of Engel and Nagel for example, it also follows from standard theory that the perturbed system $$ v_\lambda'(t) = (A + \lambda) v_\lambda(t), \quad v_\lambda(0) = x_0$$ also admits a unique classical solution for each $\lambda \in \mathbb C$. It is easy to see that this solution is given by $v_\lambda(t) := \mathrm e^{\lambda t} u(t)$. In particular, the operator $A + \lambda$ generates the so called rescaled semigroup $(\mathrm e^{\lambda t}T(t))_{t \geq 0}$, which is again strongly continuous!
Applying these general observations to your special case yields the desired result. I hope that helps :-)