For a group $G$ acting on some space $X$ we know there is a orbit-stabilizer theorem.
My question is does this formula holds for monoid action?
I think this formula may not hold, as inverse do not exist for all elements. Though I am not sure.
My second question is what information do we get from monoid action? How it is useful?
On
Monoid actions lead to the notion of a transformation monoid.
An interesting case is the following. A monoid $M$ acting on itself by right multiplication defines the (right) Cayley graph of $M$, having $M$ as set of vertices and edges of the form $(u, v, uv)$, where $u, v \in M$. The orbit of an element $m$ is the right ideal $mM$ and its (right) stabilizer is the set $\text{Stab}(m) = \{x \in M \mid mx = m\}$. The orbit-stabilizer theorem does not hold anymore, even if $M$ is finite and commutative. For instance, if $M = \{1, a, \dotsm, a^n \}$ with $a^{n+1} = a^n$, then for $m = a^{n-1}$, one gets $mM = \{a^{n-1}, a^n\}$ and $\text{Stab}(m) = \{1\}$, whence $|\text{Stab}(m)||mM| = 2$ although $|M| = n + 1$.
A systematic study of finite transformation monoids (and more generally of transformation semigroups) can be found in [1, 2]. There is also a large literature on the subject, notably in connection with algebraic automata theory. For a more advanced treatment, see [3].
[1] S. Eilenberg, Automata, languages, and machines, Vol. B, Academic Press, New York, 1976.
[2] G. Lallement, Semigroups and Combinatorial Applications, Wiley and Sons, 1979.
[3] J. Rhodes, B. Steinberg, The $q$-theory of Finite Semigroups, Springer, Monographs in Mathematics, 2009
While orbit-stabilizer is usually presented in its numerical fashion, I just want to mention it also has a "categorified" version that comes at no extra cost: any orbit is isomorphic to the coset space of a corresponding (conjugacy class of) stabilizer. Indeed, the map $G\to Gx$ given by $g\mapsto gx$ has fibers which are cosets of the stabilizer $S_x$, so the fibers form the coset space $G/S_x$ and the corresponding map $G/S_x\to Gx$ is an isomorphism in the category of $G$-sets.
Anyway, in order to explore the situation with monoids, the first thing we must do is transport the relevant definitions. It's clear what a stabilizer should be, but I see at least three "orbits":
An orbit is $Mx=\{mx:m\in M\}$ for some $x\in X$. In this sense, orbits can strictly contain each other, there needn't be maximal or minimal orbits, and $My$ may not be $Mx$ even if we choose $y\in Mx$.
An orbit $\Omega$ is nonempty and minimal with respect to the property $M\Omega=\Omega$. In this sense, elements of $X$ needn't all be in orbits, and orbits of this kind needn't exist at all. It's possible for orbits to exist in the previous sense of (1), even maximal such orbits, without these orbits existing. A weaker definition would have $M\Omega\subseteq \Omega$.
One can form the action graph whose vertices are elements of $X$ and whose directed edges are of the form $(x,mx)$ with $x\in X$ and $m\in M$. Then orbits in this third sense are the vertex sets of connected components of this graph. In this case, orbits partition $X$.
If one wants to transport the statement of orbit-stabilizer, one needs a map from $M$ to the orbit, so the best idea for stating an OS theorem would be definition (1). While definition (3) is the most elegant I think, definitions (2) and (3) don't relate to stabilizers as far as I can see anyway. There's still an equivariant surjection $M\to Mx$ via $m\mapsto mx$. Certainly the fiber of the element $x$ is the stabilizer $S_x$, but many things go wrong with this picture:
This pretty much obliterates the prospect of an orbit-stabilizer for monoid actions.
Two prime examples to explore: (i) $\Bbb N$ acting on itself or on $\Bbb Z$, and (ii) the finite cyclic nongroup monoids $\langle x|x^n=x^m\rangle$ acting on cyclic groups $\langle x|x^m=x^0\rangle$ (with $n>m>1$).