Often in physics we have integrals like the following containing delta functions inside derivatives.
Eventhough I know the 'correct' way to compute this integral is to integrate by parts until the delta function is alone and only then see that it gives this result, in this case it also works to compute the integral by simply "removing the integral and dx' and replacing all x' with x".
I was wondering if this 'trick' only works with delta functions that are on their own, and with delta functions who can be integrated by parts like this but with 0 boundary conidtions, or whether this trick of simply "removing the integral and replacing all x' with x " works more generally.
You are overthinking it. Due diligence: First note you want all derivatives in your active variable, namely x'.
Use the evident identity (prove it!), $$ (\partial_x + \partial_{x'}) ~ \delta (x-x')=0. $$
You then have $$ \int \!\! dx' ~~\partial_x \delta (x-x') ~ \partial_{x'} \psi(x')= -\int \!\! dx' ~~(\partial_{x'} \delta (x-x') )~ \partial_{x'} \psi(x')\\ = \int \!\! dx' ~~ \delta (x-x') ~ \partial_{x'}^2 \psi(x') =\partial_x^2\psi(x'). $$