Lower bound ${\bf tr}(XY) - {\bf tr}(XZY)$ by ${\bf tr}(X)$ provided $X, Y, Z$ all real symmetric

Question

Suppose $X, Y, Z$ are all real square, symmetric matrices and in particular $X, Y$ are positive definite. I am interested in a tighter lower bound of the quantity \begin{align*} {\bf tr}(XY) - {\bf tr}(XZY). \end{align*} By Von Neumann's trace inequality, we can simply devise a bound as follows: \begin{align*} {\bf tr}(XY) - {\bf tr}(XZY) \ge \lambda_{\min}(Y) {\bf tr}(X) - \|ZY\|_2 {\bf tr}(X) \ge \lambda_{\min}(Y) {\bf tr}(X) - \|Z\|_2\|Y\|_2 {\bf tr}(X), \end{align*} where $\|\cdot\|_2$ is the operator norm (largest singular value). The unsatisfying part for me is that we have $\lambda_{\min}(Y)$ and $\lambda_{\max}(Y)$ appearing. Intuitively, the equality of ${\bf tr}(XY) \ge \lambda_{\min}(Y) {\bf tr}(X) $ holds when the spectrum of $Y$ is identical and should allow us to replace $\|Y\|_2$ by $\lambda_{\min}(Y)$.