A congruence on a semigroup $S$ is an equivalence relation $\sigma\subseteq S\times S$ that respect to the multiplication. In other words $$(a,b), (c,d)\in\sigma \implies (ac, bd)\in\sigma. $$ Given a such congruence the quotient $S/\sigma$ has a well defined semigroup structure $[a][b]=[ab]$ and a quotient map $S\to S/\sigma.$
Now suppose $S$ is an inverse semigroup. Then $[a^{\star}]$ act as an inverse of $[a],$ but I do not see a reason for it to be unique. First I tried to show that if $b\in S$ has the property that$(a, aba), (b, bab)\in\sigma$ then $(a^{\star}, b)\in\sigma,$ but this doesn't seems to work.
- On the other hand, this seems bit strange. What am I doing wrong here?
- If $S/\sigma$ is merely a regular semigroup, what conditions on $\sigma$ would force it to be an inverse semigroup?
Any quotient of an inverse semigroup is an inverse semigroup. An easy way to prove this is to use the following characterisation:
Theorem. A semigroup is an inverse semigroup if and only if it is regular and its idempotents commute.
You have already shown that if $S$ is inverse, then $S/\sigma$ is regular. It remains to show that the idempotents of $S/\sigma$ commute. One needs the following lemma (see Lemma 2.4.3 in [1])
Lemma. Let $[a]$ be an idempotent in $S/\sigma$. Then there is an idempotent $e$ in $S$ such that $[e] = [a]$.
Proof. Since $[a]$ is idempotent, $(a, a^2) \in \sigma$. Let $x$ be an inverse of $a^2$ and let $e = axa$. Then $$ e^2 = a(xa^2x)a = axa = e $$ and, modulo $\sigma$ $$ e = axa \equiv a^2xa^2 = a^2 \equiv a $$ so $(e,a) \in \sigma$. $\quad\blacksquare$
It is now easy to finish the proof of the theorem. Let $[a]$ and $[b]$ be idempotents in $S/\sigma$. By the lemma, there exist idempotents $e$ in $f$ in $S$ such that $(e,a) \in \sigma$ and $(f,b) \in \sigma$. It follows that, modulo $\sigma$, $$ ab \equiv ef = fe \equiv ba $$ and thus $[a][b] = [b][a]$.
[1] John M. Howie, Fundamentals of Semigroup Theory, Oxford, Oxford University Press, coll. « London Mathematical Society Monographs. New Series » (no 12), 1995, x+351 p. (ISBN 0-19-851194-9, Math Reviews 1455373).