I have tried to prove the following problem:
Suppose $G$ is a connected matrix Lie group. Then the center of $\mathfrak{g}$ which is $\ker \textrm{ad}$ is equal to the Lie algebra of $Z(G)$.

Recall that $\textrm{ad} : \mathfrak{g} \to \text{gl}(\mathfrak{g})$ is the map that sends $X \in \mathfrak{g}$ to $\textrm{ad}_X$, where $\textrm{ad}_X$ is defined by
$$\textrm{ad}_X(Y) = [X,Y].$$ Similarly $\textrm{Ad}_A$ is the map such that for all $X \in \mathfrak{g}$, we have $\textrm{Ad}_A(X) = AXA^{-1}$. Now I have proven the problem above as follows: If $Y \in \ker \textrm{ad}$, then certainly for all $t \in \Bbb{R}$ we have $\textrm{ad}_{tY} = 0$. Now we have given any $x \in G$ that
$$\begin{eqnarray*} e^{tY}xe^{-tY} &=& \textrm{Ad}_{e^{tY}}(x) \\ &=& e^{\textrm{ad}_{tY}}(x) \\ &=& x \end{eqnarray*}$$
by assumption and since this holds for all $x$, we conclude that $e^{tY} \in Z(G)$ for all $t$ and so $Y$ is in the Lie algebra of $Z(G)$.
For the converse, if $Y$ is in the Lie algebra of $Z(G)$, then $e^{tY} \in Z(G)$ for all $t$ and so given any $x \in G$, $e^{tY}xe^{-tY} = x$. We now have $e^{\textrm{ad}_{tY}}x =x$ for all $x$, which implies that $e^{\textrm{ad}_{tY}} = I \in \textrm{GL}(\mathfrak{g})$. Taking the derivative at $t = 0$ implies that $\textrm{ad}_Y = 0$ and so $Y \in \ker \textrm{ad}$.
However: I have nowhere used the assumption of connectedness of $G$. What have I done wrong?
Since $\operatorname{ad}_{tY}$ acts on the Lie algebra of $G$ (only), you cannot expect $e^{\operatorname{ad}_{tY}}=\operatorname{Ad}_{e^{tY}}$ to hold as identity of functions defined beyond the connected component $G_0$ of the identity of $G$. Indeed if you think of $G=O(2,\mathbf R)$ where $G_0$ is the commutative group of $SO(2,\mathbf R)$, you can see that $\operatorname{ad}_{tY}=0$ for any $Y$, but $\operatorname{Ad}_{e^{tY}}$ is conjugation by a rotation, which does not fix reflections (unless $e^{tY}=\pm \mathrm{Id}$).
In fact now that I reread your question carefully, I see that you defined $\operatorname{Ad}_A$ to be a function defined on the Lie algebra $\mathfrak{g}$, not on $G$, so this makes your argument that applies it to $x\in G$ invalid in the first place. But my point is that although you can define $\operatorname{Ad}_g$ (maybe with a different notation) to be a conjugation by $g$ in $G$, you cannot expect $\operatorname{ad}_{tY}$ to tell you what any such $\operatorname{Ad}_g$ does beyond $G_0$.