Just as the title says:
Can there be an infinite set S whose elements all contain S?
If so, how is it called?
An example of a finite set could be the set of all Humans, where each human h knows about all other humans (where 'knowing' means being part of a set).
If there are infinite things in reality, and all things interact with each other, it could also be an example of the former infinite set.
Does this have a name in mathematics?
In ZFC, the answer is of course no, by the axiom of foundation.
But under AFA (see here) the answer is yes. Consider the directed graph whose vertex set is $\mathbb{N}\cup \{S\}$. There is an edge from $S$ to $n$ and from $n$ to $S$ for each $n$, and there is an edge from $n$ to $m$ if and only if $m<n$. This is a pointed (at $S$) accessible directed graph.
By AFA, there is a set $S$ such that the membership relation on the transitive closure of $S$ is the graph described above. Now one can show that each $n$ corresponds to a distinct set, and they are all members of $S$, so $S$ is infinite. And every element of $S$ contains $S$, as desired.