What does it mean for a finite-rank operator to be "smooth"? How can one finite-rank operator be smoother than another? For me, a finite-rank operator on $L^2(\mathbb R^d)$ has an expression $$Q=\sum_{j=1}^N\langle \phi_j,\cdot\rangle\psi_j$$ for $(\phi_j)_{j=1}^N$ and $(\psi_j)_{j=1}^N$ orthonormal.
So a guess is saying that $(\psi_j)$ have to be smooth as well as in $L^2(\mathbb R^d)$?