Given concrete category $\mathcal{C}$, does every isomorphism (arrow) $f:X\rightarrow Y$ correspond to a bijection (function) $f:X\rightarrow Y$. I know that bijections are precisely the functions having both-sided inverse in $\operatorname{Set}$.
So basically given an iso $f$ there is its inverse $g$, then taking it by the faithful functor, we have $F(f)F(g)=F(fg)=F(id)=id_{Sset}$, and similarly from the other side, but functions having both-sided inverse are precisely bijections in $\operatorname{Set}$. Does this suffice?
If $F\colon \mathcal{C}\to \mathbf{Set}$ is any functor and $f$ is an iso in $\mathcal{C}$, then $Ff$ has inverse $F(f^{-1})$. This is an inverse in $\mathbf{Set}$, as you've noticed, and the isos in that category are precisely the bijections.
However, what I suspect you want to ask is if in every concrete category, an arrow is an iso iff its underlying morphisms of sets is an isomorphism. This is definitely not true in general (good exercise).