I've learned that $\forall x \forall y \phi$ is equivalent to $\forall y \forall x \phi$, and the same is true for $\exists x \exists y$. However $\exists !x \exists !y$ is not equivalent to $\exists !y \exists !x$. I do not see how this makes sense intuitively.
Additionally, I've been evaluating these expressions as "for all variables x and for all variables y", "there exists variables x and y", so this would be "there only exists one x, y".
It seems that my way of thinking about this is incorrect.
How should I be evaluating these?
To start things off, let's whip up a simple-to-understand example showing that the sort of switching you have in mind can't possibly be legal in general (it sounds like you already know this, but other readers might benefit from its explicit inclusion here):
Imagine a large dining room with lots of tables, where all but one of the tables have multiple chairs and a single table has a single chair. Let $\varphi(a,b)$ be the formula "$a$ is a chair at table $b$." It may well be the case that exactly one table has exactly one chair at it. However, barring some very surprising chair-related shenanigans, it should not be the case that exactly one chair belongs to exactly one table: presumably every chair belongs to exactly one table. In any reasonable instance of the above situation, $\exists!x\exists!y\varphi(x,y)$ should be false but $\exists!y\exists!x\varphi(x,y)$ should be true.
OK, now we know that there's got to be something wonky going on; let's find it.
As is often the case, the culprit lies in the set of our human-linguistic-efficiency-oriented foibles. Despite the notation we use for it, "$\exists!$" should really be thought of as a mixed-quantifier expression: when we write out $$\exists !x\varphi(x)$$ in terms of the basic quantifiers $\exists$ and $\forall$, what we get is $$\exists x\forall x'[\varphi(x)\wedge(\varphi(x')\rightarrow x=x')]$$ (or something similar). In particular, this means that a quantifier prefix like $$\exists!x\exists!y$$ should really be thought of as $$\exists x\forall x'\exists y\forall y',$$ and the switching you have in mind would require passing the $\forall x'$ through the $\exists y$ and passing the $\exists x$ through the $\forall y'$. Each of these two maneuvers is of course illegal in general. Incidentally, the discussion here is relevant.
Finally, as a fun exercise let's dive into the parenthetical remark above. We can also interpret "$\exists!$" as having the form "$\forall\forall\exists$" (phrasing this precisely takes a bit of work, but the idea should be intuitively clear):
Meanwhile it's not hard to show that we cannot interpret $\exists!$ in a purely-existential, purely-universal, or $\forall\exists$ way. So the pair of interpretations we've seen above - $\exists\forall$ and $\forall\forall\exists$ - is in a precise sense optimal.