Sets are defined in such a way that $\{a,a\}$ is the same as $\{a\}$, and $\{a,b\}$ is the same as $\{b,a\}$. By contrast, the ordered pair $(a,a)$ is distinct from $(a)$, and $(a,b)$ is distinct from $(b,a)$.
Intuitively, it would seem useful to draw a distinction between two collections if they are ordered differently, or if one collection has a different number of copies of an element to the other. For instance, this would mean that the collection of prime factors of $6$ would be different to that of $12$. However, it is the set, rather than the tuple, that is chosen as the primitive object. Why is it useful for the foundations of mathematics that sets have very little "structure", and would their be any difficulties in choosing tuples to be the primitive object instead?
The simpler a kind of object is, the more impressive its expressive power becomes. Sets are more compelling than tuples as an answer to the question, "How structurally simple can we make the foundations of mathematics?" Now that may not be a question one cares about - in particular, structural simplicity is often at odds with actual usability, and moreover one might reasonably reject simplicity as an inherent virtue in this context at all - but it is a meaningful question. And the early history of modern logic (especially around logicism) highlights this question.
So is there a problem with taking tuples - or even more complicated objects - as our "ground objects" for founding mathematics? Well, it depends on exactly what our desiderata of that foundational project are. Certainly there's no fatal obstacle, in the sense that from a technical perspective a tuple-based approach will be as safe and expressive as a set-based approach, but we might still have additional reasons to prefer the sets-based approach.
For what it's worth, I believe that simplicity-of-base-objects as a foundational criterion has lost a lot of steam over the last few decades, especially outside of the mathematical logic community: the notion of structural foundations of mathematics has grown in popularity.