I am doing a simple exercice and I think that either the book's solution is wrong or I misunderstood the problem.
Here is the problem,
平面上で次の曲線又は直線で囲まれる図形の面積を求めよ。
極座標系について、曲線 $r = a(1+2\cos \theta)$, $(0 ≦ \theta ≦ \frac{2\pi}{3})$, $(a > 0)$と直線 $\theta = 0$.
and its translation:
In the plane, please find the area enclosed by the curves or straight lines.
Consider the curve $r = a(1+2\cos \theta)$, $(0 \leq \theta \leq \frac{2\pi}{3})$, $(a > 0)$ and the straight line $\theta = 0$.
The book's answer is: $$S = \frac{a^2}{2}(\pi + \frac{7}{8}\sqrt{3})$$
Here is what I have done:
The line $\theta = 0$ is just the same as the $x$-axis so I can safely ignore it in further calculations according to the graph of the function $r$ (that can be seen here).
So I just need to calculate: $$ I = \frac{2S}{a^2} = \int_0^{\frac{2\pi}{3}}(1+2\cos\theta)^2\mathrm{d}\theta$$. Which gives: $$ S = \frac{a^2}{2}(2\pi + \frac{3}{2}\sqrt{3}) $$
Here is how I worked out the integral:
$$ \begin{align} I &= \int_0^{\frac{2\pi}{3}} 1 + 4cos\theta + 4\cos^2\theta\mathrm{d}\theta \\ I &= \frac{2\pi}{3} + 4\left[ \sin\theta \right]_0^{\frac{2\pi}{3}} + 2\left[ \theta + \frac{\sin2\theta}{2} \right]_0^{\frac{2\pi}{3}} \\ I &= \frac{2\pi}{3} + 4(\frac{\sqrt{3}}{2}) + 2(\frac{2\pi}{3}-\frac{\sqrt{3}}{4}) = 2\pi + \frac{3}{2}\sqrt{3} \end{align}$$
This only shows what the book's answer means. It does not mean that the op's answer is wrong because the given curve is $\left[0,\dfrac{2 \pi}{3}\right]$ (blue one), the red one is $\left[\pi,\dfrac{4 \pi}{3}\right]$