I have been commanded on homework to find a non-bijective isomorphism in a category whose objects are sets, whose morphisms are set maps, and composition is the usual function composition. So our category is a subcategory of the category of Set.
But, I think that such an isomorphism is in fact impossible to find.
Proof
Let $\mathcal C$ be a category of sets with morphisms being set maps and composition being the usual set composition. If $\mathcal C$ is the empty category, we have no isomorphisms, as we have no morphisms. Thus $\mathcal{C}$ contains at least a single object.
Take $A,B \in \mathcal C$ and assume we have an isomorphism $f \in Mor(A,B)$.
(Note that $A=B$ is allowed).
Since $f$ is an isomorphism, there exists $g \in Mor(B,A)$ such that $f \circ g = 1_B$ and $g \circ f = 1_A$. We show that $f$ must in fact be a bijection.
Injective: Take $a_1$ and $a_2$ in $A$ and suppose that $f(a_1) = f(a_2)$. We then have the following chain of equality: $$(g\circ f)(a_1) = 1_A(a_1) = a_1 = a_2 = 1_A(a_2) = (g\circ f)(a_2).$$
Hence we have that whenever $f(a_1) = f(a_2)$, we have $a_1 = a_2$. So $f$ must be injective.
Surjective: Take $b$ in $B$. We know that the morphism $g$ takes elements of $B$ to $A$. Hence $g(b) \in A$. We then have $$f(g(b)) = (f\circ g)(b) = 1_B(b) = b.$$
Thus $f$ maps $g(b) \in A$ to $b$ and $f$ is therefore surjective.
We can therefore conclude that any isomorphism in $\mathcal C$ must be bijective.
Given the hint that the identity morphisms can be weird in ((sets)), we can proceed as follows. Let ((sets)) consist of just the one set $\{0,1\}$ and the one map, from $\{0,1\}$ to itself that sends both 0 and 1 to 0. This one map is then the identity map, composition is as usual, and this identity map is a non-bijective isomorphism.