Consider the following integral :
$$ \int_{\tau'}^{\tau''} d\tau f(\tau) \frac{d}{dt}[\delta(t-\tau)] $$
We can put out of the integral the derivative with respect to time $t$ and thus we have :
$$ \int_{\tau'}^{\tau''} d\tau f(\tau) \frac{d}{dt}[\delta(t-\tau)]=\frac{d}{dt}[f(t)] $$
But I also have :
$$ \frac{d}{dt}[\delta(t-\tau)] = -\frac{d}{d \tau}[\delta(t-\tau)]$$
And if I plug this I have :
$$ \int_{\tau'}^{\tau''} d\tau f(\tau) \frac{d}{dt}[\delta(t-\tau)] = - \int_{\tau'}^{\tau''} d\tau f(\tau) \frac{d}{d\tau}[\delta(t-\tau)]$$
And after doing an integration by part I will have the same result as before plus a surface term.
So the two way of computing the integral don't give the same results.
Where is the problem in what I have done ?
The "surface term" is 0 due to the delta. That is, the term from the integration by parts $$ \left [ f(\tau) \delta(t - \tau) \right ]_{\tau'}^{\tau''} = f(\tau'')\delta(t-\tau'') - f(\tau')\delta(t-\tau') = 0 $$ and you recover the expression obtained through the other method. In the case where $t=\tau''$ or $t=\tau'$, it depends on how you define the Dirac-delta, in a physics sense or in the sense of distributions. Often in physics one defines $$ \int_0^a f(t) \delta(t) \mathrm{d}t = \frac{1}{2} f(0) $$ for example, but then the concept of $\frac{\mathrm{d} \delta}{\mathrm{d} t}$ is another matter altogether.