So a lot of people are like $\mathbb{Z}$ has a unique factorization up to units...but that doesn't make sense to me because (for example), $(-1) \cdot (-1) \cdot 2 \cdot 3$ should be a factorization; but then $2 \cdot 3$ is also a factorization, so is it really unique?
I guess I'm confused what unique up till units means (memes)
On
Unique factorization is often not formally defined, because the terminology to write what it actually means is laborious. It is often taught in a kind of “you know what we mean” way, rather informally.
Formally, we’d have to talk about the products of finite multisets of primes, for example.
“Up to units” is similarly sometimes vague.
How to make it more precise
Given an integral domain $R,$ consider $M=R\setminus\{0\}/\sim$ where $a\sim b$ if $a=bu$ for some unit.
Then we get a multiplication in $M$ that is inherited from multiplication by $R.$ Namely, since $a\sim b, c\sim d$ me and $ac\sim bd,$ we can define $$[a]\cdot_M[b]=[a\cdot_R b].$$
$M$ can be seen as representing the “multiplicative structure of $R,$ up to units.”
Then $M$ is a commutative monoid, and if we can show unique factorization there, we say $R$ is a unique factorization domain, or has unique factorization 7pm to units.
In $\mathbb Z,$ $M$ is isomorphic to $(\mathbb Z^+,\cdot).$ In other rings, $M$ is not isomorphic to a sub-monoid of $(R,\cdot).$
Another reason to study the multiplication up to units
Divisibility in $R$ is a pre-order, but divisibility in $M$ becomes a more common thing, a partially ordered set. In unique factorization domains, that partially ordered set has nice properties, like being a distributive lattice.
We generally don’t like studying pre-orders, we like to study the related partial order.
"Up till units" means exactly what you observed, and if you are into math you can see it being used while proving the unique factorization side of the Fundamental Theorem of Arithmetic (see this under Uniqueness without Euclid's lemma); while dividing you are left just with units and you need to rule them out for the proof to work.
Since I do not know how fond you are into elementary number theory - and if you are not that link would probably be a tad cumbersome - it should suffice to note that units would be an issue for the reason you stated, and thus need to be excluded from the "unique" adjective; they are not considered, together with the position of the primes. On the other side, considering them would create issues even with non-negative numbers, think about $7=1\cdot1\cdot1\cdot7$ or $7=7\cdot1$.