I want to find functions $f: A \to B$ and $g: B \to A$ such that $g(f) = i_A$ but $f(g) \neq i_B$.
Is this possible? An exercise for one of my classes is to actually prove that if $g(f) = i_A$, then $f(g) = i_B$.
I've found functions where $f(g(x)) = x$ for most $x$, but failed for an exceptional choice of $x$. Would that be a counter example?
Are there examples where $g(f(x)) = x$ for all $x \in A$ and $f(g(x)) \neq x$ for all $x \in B$?
The fact that $g(f)=i_A$ tells you that $f$ must be injective and $g$ must be surjective. If you want a counterexample, you must look at functions $f$ and $g$ such that
An example can be $f:\{0\}\to \{0,1\}$, $g:\{0,1\}\to \{0\}$ with $$f(0)=0, g(0)=0, g(1)=0$$
then $g(f)=i_{\{0\}}$ since $g(f(0))=g(0)=0$ but $f(g)\neq i_{\{0,1\}}$ since $f(g(1))=f(0)=0$.