A semi-complement (for the purpose of this question) on a poset $P$ is an involution (self-inverse w.r.t. to composition) $\neg : P\to P$, such that:
$$\neg\left(\bigvee_{i\in I} x_i\right) = \bigwedge_{i\in I} \neg x_i$$ $$\neg\left(\bigwedge_{i\in I} x_i\right) = \bigvee_{i\in I} \neg x_i$$
whenever these suprema / infima exist.
Obviously every Boolean algebra has a semi-complement. Here are some examples, where $P$ is not a Boolean algebra:
- the poset $[0\,..\infty]$ with $\neg x := 1/x$, where $\neg 0 = \infty$.
- using the previous example: the poset of functions $X\to [0\,..\infty]$ with $\neg f = 1/f$
- the poset of measurable functions $E \to [0\,..\infty]$ for some measurable space $(E,\mathcal{E})$ with $\neg f = 1/f$
- the "Cheng space construction": Let $L$ be a Heyting algebra and let: $$P := \{(x,y) \in L \times L^{\operatorname{op}} : x\wedge y =\bot\}$$ Then, let $\neg(x,y) = (y,x)$. (I believe this should work possibly with some tweaking, inspired by this)
What other examples are there of semi-complements?
Generalizing your examples: they are related to lattice ordered groups, where the involution is $x\mapsto x ^ {-1}$. It's not hard to prove that in any lattice ordered group, $x\mapsto x ^ {-1}$ meets your criteria yet the lattice is not Boolean (l-groups cannot be bounded, for one thing).
This generates wide variety of examples, e.g. $G =(\mathbb Q ^ n, +)$. Or a more exotic one: consider some totally ordered set $\Omega$ and let $G$ be the group of all order preserving automorphisms on $\Omega$ where we define $f\leq g\equiv\forall x: f(x)\leq g(x)$.
Here's an interesting question: suppose we are working with an l-group and some function $x\mapsto x ^*$ meets your criteria. Must $x ^*= x ^ {-1}$? I think the answer is "yes", but I am not sure.
EDIT: answering your comment from below.
I agree that $0,\infty$ are problematic in your first example. I think the most obvious way to relate them to a group is to just drop the problematic elements and look at $(\mathbb R ^ +,\cdot)$, but you could try something like the hyperreal numbers instead. In any case, let's assume you have figure out some way to handle this – then I claim your second example can handle its problematic elements in the same way. Let's say your first example is some group $G$.
Observe that in the case that $| X | = 1$ your second example is exactly the same as the first (do you see why?). More generally, once you handle $0$ and $\infty$ there's a natural isomorphism between your second example and $G ^ {| X |}$ under component-wise multiplication given by $\varphi (f) = (f(x_0), f(x_1),\dots)$. (I.e. your second example is just the direct product of the first example some number of times.)
So, to elaborate on your (correct) point that there are more than 2 problematic elements: if $X =\{1, 2\}$, there are an infinite number of functions $f_z$ where $f_z(1) = 0, f_z(2) = z$. But clearly $\neg f_z(1) =\neg 0$, where $\neg 0$ is whatever you defined the inverse to be in your first example.
This is why, even though there are an infinite number of elements "without inverses", you only need to handle the two cases from your first example.