More clearly, my question is:
Let $G$ be a finite group, there are some cyclic subgroups $H_i$ of $G$,
$\text{s.t. } G=\bigcup H_i,H_i \cap H_j = \{ e \}(i \ne j)$.
I know if $G$ is infinite, it is wrong. (I consider the example $(\mathbb Z,+)$)
I believe when $G$ is finite, it is wrong, too. But I find it difficult for me to construct counter examples.
Thanks to @Lord Shark the Unknown.
A quaternion group is exactly a counter example.
A quaternion group has only 5 cyclic subgroup:
$H_1=\{ 1 \}\\H_2=\{ 1, -1 \}\\H_3=\{1, i, -1, -i \}\\H_4=\{1, j, -1, -j\}\\H_5=\{1, k,-1 ,-k\}$
So if let the union is $G$, then $i,j \in G$. We must choose $H_3$ and $H_4$, but $H_3 \cap H_4 \not= \{ 1 \}$.
I do not really understand what a quaternion group really is, but I believe it very beautiful.