How can the following integral can be solved $$I(t_1,t_2)=\int_0^{t_1}dt'e^{-a(t_1-t')}\int_0^{t_2}dt''e^{-a(t_2-t'')}\delta(t''-t')$$ where there are no assumptions regarding $t_1$ or $t_2$ (both cases of $t_1>t_2$ or $t_2>t_1$ are valid).
are there any limitations for using the Dirac delta here?
First I do the inner integral: $$ \int_0^{t_2} dt'' \, e^{-a(t_2-t'')} \, \delta(t''-t') = \int_{-\infty}^{\infty} dt'' \, \chi_{[0, t_2]}(t'') \, e^{-a(t_2-t'')} \, \delta(t''-t') = \chi_{[0, t_2]}(t') \, e^{-a(t_2-t')} $$ Here, for any set $A$, $\chi_A(t) = 1$ if $t\in A$, $\chi_A(t)=0$ otherwise.
Then the outer integral: $$ I(t_1,t_2) = \int_0^{t_1} dt' \, e^{-a(t_1-t')} \, \chi_{[0, t_2]}(t') \, e^{-a(t_2-t')} \\ = e^{-a(t_1+t_2)} \int_0^{t_1} dt' \, e^{2at'} \, \chi_{[0, t_2]}(t') \\ = e^{-a(t_1+t_2)} \int_{-\infty}^{\infty} dt' \, \chi_{[0, t_1]}(t') \, e^{2at'} \, \chi_{[0, t_2]}(t') \\ = e^{-a(t_1+t_2)} \int_{-\infty}^{\infty} dt' \, \chi_{[0, \min(t_1,t_2)]}(t') \, e^{2at'} \\ = e^{-a(t_1+t_2)} \int_{0}^{\min(t_1,t_2)} dt' \, e^{2at'} \\ = e^{-a(t_1+t_2)} \left[\frac{1}{2a} e^{2at'} \right]_{0}^{\min(t_1,t_2)} \\ = \frac{1}{2a} e^{-a(t_1+t_2)} \left[e^{2a\min(t_1,t_2)} -1 \right] $$