Let $X,Y$ be compact and $\tau: Y\to X$ be continuous. If $A:C(X)\to C(Y)$ such that $(Af)(y)=f(\tau(y))$ is a linear and continuous operator, then show that the adjoint operator $A^*:M(Y)\to M(X)$ is given by $$(A^*\mu)(U) = \mu (\tau^{-1}(U))$$ for every Borel subset $U$ of $X$ and every $\mu \in M(Y)$.
My attempt: I just know for $f\in C(X)$ , $\mu \in M(Y)$;
$$(f,A^* \mu) =(Af,\mu)=\int (Af)(y)d\mu(y)=\int f(\tau(y))d\mu(y)$$
Please help me