This is an example from my book. I attempted it on my own and I did it differently than the authors but I'm not sure if the way I did it is correct. The following is what I did (note its not formal):
Observe that, $$ \begin{align} \lvert x^2+xy+y-3\lvert\:&=\lvert x^2-x+x-1+xy+y-2\lvert\\ &=\lvert x(x-1)+x-1+xy+y-2\lvert\\ &\leq \lvert x\lvert\lvert x-1\lvert+\lvert x-1 \lvert+\lvert xy\lvert+\lvert y-2\lvert\\ &\leq \lvert x\lvert\lvert x-1\lvert+\lvert x-1 \lvert+\lvert x\lvert\lvert y\lvert+\lvert y-2\lvert\\ &< 2\delta+\delta+4+0\\ &=3\delta +4 \end{align} $$ So what I did is I let $\delta=\frac{\epsilon-4}{3}.$ Is this wrong? I know $\epsilon$ can't depend on $x$ or $y$ but the only reason why I think this doesn't work is because it takes away the purpose of $\epsilon$ being arbitrarily small. But then I think I've seen similar things done, I'll have to check my books.
Generally speaking, what are the "rules" when one is letting $\delta$ be something in the form where $\epsilon$ is used? For example, letting $\delta=\frac{x\epsilon}{3}$ would be wrong. Any other ones?
As I mentioned in the comments, $\delta = \frac{\epsilon - 4}{3}$ will not work because $\delta < 0$ whenever $\varepsilon < 4$. To establish a limit, you need a valid $\delta$ for all $\varepsilon > 0$, including (say) $\varepsilon = 1$. And, there is no $\delta > 0$ such that $$\|(x, y) - (1, 1)\| < \delta \implies |x| |x - 1| + |x - 1| + |x| |y| + |y - 2| < 1.$$ It means you've taken a wrong turn up a blind alley. Your inequalities, though absolutely valid, have gone too far and failed to yield something useful for the problem.
Using triangle inequality is a good idea, but here's a rule of thumb when using triangle inequality in a limits proof: make sure each term can be made small individually. For example, $|x| |x - 1|$ is fine, because $|x|$ is bounded and $|x - 1|$ is always less than $\delta$, which we can make as small as we like. However, $|x||y|$ and $|y - 1|$ are not OK, since making $x$ and $y$ really close to $1$ will make these quantities really close to $1$, which means you really can't force the expression to be significantly smaller than $2$. Don't forget that you're summing positive quantities here, so if one term spoils things by being at least some positive number, you can't make the whole expression sum to an arbitrarily small number.
Instead of splitting $|xy + y - 2| \le |xy| + |y - 2|$, instead try splitting $|xy - 1| + |y - 1|$. This now has the benefit that both terms can be made as small as you like by limiting $\delta$. You can then deal with $|xy - 1|$ in much the same way as you did with $|x^2 - 1|$ (by adding and subtracting $x$).