Suppose we have random variable $X$ and $Y$ where $Z$. Then using reverse triangle inequality we have that \begin{align} \|X-Y\|_2 \ge \left|\|X\|_2-\|Y\|_2 \right| \end{align} However, I can come up with an example such that $\|X-Y\|_2<\infty$ but both $\|X\|_2,\|Y\|_2=\infty$. For example, let $Y=X+Z$ where $Z$ is any random variable that has a finite second moment. For example, pick $Z$ such that it is 1 and $X$ is Cauchy distribution.
So, we can have a case \begin{align} 1 \ge | \infty-\infty| \end{align}
Is this a problem? Or we just assume that $| \infty-\infty|=0$.
P.S. $\|X\|_2=(E[X^2])^{1/2}$