Suppose that $\mathcal{I}$ is the category where objects are sets and morphisms are injective functions.
Every epimorphism is an isomorphism. Suppose $f : A \to B$ is an epimorphism in $\mathcal{I}$. Since $f$ is injective, it must be a monomorphism in $\mathcal{I}$. Consider $g, h : B \to C$. By definition, $f \circ g = f \circ h \implies g = h$ and $g \circ f = h \circ f \implies g = h$. So if there is an inverse for $f$, then it must be unique. To construct the inverse for $f$, can we say that if $f(x) = y$, then $f^{-1}(y) = x$? This seems to be a proper inverse.
Does $\mathcal{I}$ have all equalizers? Coequalizers? To show that $\mathcal{I}$ has all equalizers, consider morphisms $g, h : A \to B$ in $\mathcal{I}$. We can construct an equalizer for these two functions as follows. Consider the set $E = \{ x \in A : g(x) = h(x)\}$. We say that $f$ is the natural inclusion from $E$ to $A$ (note that this is indeed an injective function). Then of course $g \circ f = h \circ f$ by definition of $f$.
Epimorphism in $\mathcal I$ are surjective functions: Let $f\colon A\to B$ be an epimorphism and assume $f$ is not a surjective map, say $b\in B$ is not in its image. We can define $g_1\colon B\to B\cup\{B\}$ by letting $$g_1(x)=\begin{cases}B&x=b\\x&x\ne b\end{cases}$$ and let $g_2\colon B\to B\cup\{B\}$ be the inclusion map. Then $g_1\circ f=g_2\circ f$, contradicting $g_1(b)\ne g_2(b)$.
Hence epimorphisms in $\mathcal I$ are bijections, which obviously have an inverse function (that is of course injective).
Let $f,g\colon X\to Y$. Then we have the map $E=\{\,x\in X\mid f(x)=g(x)\,\}$ and the inclusion $\iota\colon E\to X$. This is an equalizer: Clearly $f\circ \iota=g\circ\iota$. If $m\colon O\to X$ is such that $f\circ m=g\circ m$ then - as $f,g$ are injective - clearly $m[O]\subseteq E$ and the inclusion $O\to E$ is an injective map with the desired property. As everything is injective, thsi map is clearly unique.