I am confused with the two notions.
I basically understand Hermitian connection: it is a complex analog of metric-compatible connection on $TM$, a connection that preserves the hermitian metric $h$ defined for a general complex vector bundle.
But what about unitary connection? I searched for a while but did not find a good explanation.
I think, unitary connection refers to a compatible connection on the principal $U(n)$-bundle $P\to M$ (for some $n$). If you associate with $P$ a hermitian vector bundle $E\to M$ in the standard fashion, then unitary connection on $P$ will yield a hermitian connection on $M$ and vice versa. All this should be in Kobayashi and Nomizu "Foundations of Differential Geometry".
Edit: In the context of connections on vector bundles, "unitary connection" is synonymous to "hermitian connection". Compare for instance here.