I'm reading a famous book, "Group Theory and Quantum Mechanics" written by Michael Tinkham. In Chapter 5 of this book, he introduces the relationship between $SU(2)$ and $SO(3)$ and I already understand the reason that $SU(2)$ is covering Lie group of $SO(3)$, which can be regarded as Lie group homomorphism. Then the book says, (p. 106)
Because of the homomorphism, these same matrices(= all irreducible, unitary matrices representing $SU(2)$) will serve as the representation matrices for the three-dimensional rotation group.
My question is "why?". Well, the book firstly derive all possible "irreducible, unitary" representation matrices of $SU(2)$, $U^{(j)}$, having $2j+1$ dimension. With $u\, \in $matrix Lie group $SU(2)$ and $P(u) \in $ Lie group $SU(2)$, we can write
$$P(u)f^{(j)}_m\,=\, \sum_{m'}f_{m'}^{(j)}U^{(j)}(u)_{m'm}$$
where $f^{(j)}_m$ is appropriate basis functions. Accordingly we can possibly find $2j+1$ dimensional matirces $D^{(j)}(R)$ satisfying
$$P(R)f^{(j)}_m\,=\, \sum_{m'}f_{m'}^{(j)}D^{(j)}(R)_{m'm}$$
where $R\in$matrix Lie group $SO(3)$ and $P(R)\in$Lie group $SO(3)$. Then the book says
$$D^{(j)}(R)=U^{(j)}(u^{-1})$$
and this seems to be the part corresponding to boldfaced statement above.
As mentioned, here is a Lie group homomorphism $\Phi$,
$$\Phi\,:\,P(u)\,\rightarrow\,P(R)$$ or $$u\,\rightarrow\,R$$
However I cannot have any idea that relates the existence of $\Phi$ and the fact that $SU(2)$ and $SO(3)$ share same representing matrices, $D^{(j)}(R)=U^{(j)}(u^{-1})$.
This sounds a bit like a misunderstanding. What is true is that any representation of $SO(3)$ gives rise to a representation of $SU(2)$, but not vice versa. This is easy to see if one interprets a representation as a homomorphism to a unitary group. Given a representation $SO(3)\to U(N)$ (for some $N$), you just compose with the homomorphism $SU(2)\to SO(3)$ to obtain a representation $SU(2)\to U(N)$.
For a converse, a bit more information is needed, namely that the homomorphisms $SU(2)\to SO(3)$ acutally induces an isomorphism $SU(2)/\{\pm\mathbb I\}\to SO(3)$. (This basically expresses the fact that $SU(2)$ is a two-fold covering of $SO(3)$). Now if you have a representation of $SU(2)$, it may happen that $-\mathbb I$ is represented by the identity matrix. If this is the case, then the representation defines a representation of $SO(3)$, since it descends to the quotient $SU(2)/\{\pm\mathbb I\}$. In this way, representations of $SO(3)$ are "the same" as representations of $SU(2)$ in which $-\mathbb I$ is represented by the identity. (Roughly speaking this is "half" of the irreducible representations of $SU(2)$. From your question it sounds like the book would only consider these, otherwise the condition on odd dimension of the representing matrices should not be there.) This obvious representation of $SU(2)$ on $\mathbb C^2$ does not give rise to a representation of $SO(3)$ and it is the simplest example of such a representation. This is the basis for spinors.