I've seen other posts (Inner automorphisms form a normal subgroup of $\operatorname{Aut}(G)$) about the topic of $\operatorname{Inn}(G) \simeq G/Z(G)$, but what I want to ask is a detail. When we ensure that $\operatorname{Inn}(G) \simeq G/Z(G)$, we want to say that there exists some isomorphism $F$ such that:
$$F: G/Z(G) \rightarrow \operatorname{Inn}(G), \quad F(gz) = \tau_g$$
where $g \in G, z \in Z(G)$ and $\tau_g(h) = ghg^{-1}$ for all $h \in G$
But is this $F$ really an isomorphism? I'm not pretty sure due to there is not ONE $\tau_g$ for ONE $gz$, what we have is a total of $|Z(G)| = \text{order of }Z(G)$ elements of the form $gz$ for just ONE $\tau_g$ due to $\tau_g = \tau_{gz}$ if $z \in Z(G)$
Well, to see that $F$ is a bijection, we can show that it is surjective and inyective. I will note $Z := Z(G)$.
Surjectivity: let $\tau_g(x) = gxg^{-1}$ an element of $Inn(G)$. If we now consider the class of $g$ in $G/Z$, it has image precisely $\tau_g$ via $F$, as you described.
Injectivity: suppose that $F(gZ) = F(hZ)$. Thus, $\tau_g \equiv \tau_h$ and thus
$$ gxg^{-1} = \tau_g(x) = \tau_h(x) = hxh^{-1} \ (\forall x \in G) $$
or equivalently, $h^{-1}gx = xh^{-1}g$. Therefore, $h^{-1}g \in Z$ which implies $gZ = hZ$.
Furthermore, $F(1_{Z/G}) = F(1 \cdot Z) = \tau_1 = id_G$ and $$F(gZ)F(hZ) = \tau_g\tau_h = \tau_{gh} = F(ghZ) = F(gZhZ)$$
thus proving that $F$ is not only a bijection but an isomorphism.
Intutively, what is happening is that conjugation by some $g \in G$ is 'not affected by the component of $g$ in $Z$'. By that I mean that if $g = sz$ with $z \in Z$, $\tau_{sz} = \tau_s$. So we can group elements which only differ in a traslation via an element of the center, since their conjugation will be the same: that is exactly what $G/Z(G)$ is.