Given a set $S$, say $S= \{a,b\}$, the set of permuations of $S$ is $\{[a,b],[b,a]\}$ (these are supposed to be lists like $ab$ and $ba$). Can we define a monad that captures this? This monad would be like the List monad.
The permutation monad $Perm = (P, \mu, \eta)$.
$$P:Set \rightarrow Set$$ such that, P returns the set of permutations of a given set.
$$\mu : P \cdot P \rightarrow P$$ by returning a set of permutations given by the first element in each permutation of permutations. (This I don't think works). $$\eta : 1_{Set} \rightarrow P$$ by sending a set element to it's permutation of itself.
Note that you have only defined $P$ on objects. There isn't a nice way of extending $P$ to a functor $\mathbf{Set} \to \mathbf{Set}$, however there is another category where your definition of $P$ does extend nicely. Namely, let $\mathbf{B}$ be the category of sets and bijections; then you can define $P(X)$ to be the set of permutations of $X$ and, given a bijection $f : X \to Y$, define $$P(f) : P(X) \to P(Y), \quad \sigma \mapsto f \circ \sigma \circ f^{-1}$$ Note that $P(f)$ is itself a bijection and that the assignment $f \mapsto P(f)$ is functorial, so this truly does define a functor $P : \mathbf{B} \to \mathbf{B}$.
Unfortunately, this is where the story ends. To define a natural transformation $\eta : 1 \to P$, you would need to define a bijection $\eta_X : X \to P(X)$ for each set $X$, but basic cardinality arguments demonstrate that this is impossible, since there are more permutations of a set $X$ than there are elements of $X$.