The graph of $r = 10\cos(3\theta)$ has three petals (also called "leaves" or "lobes").
The intersection of one of those petals with the circle $r = 5$ is shaded in the figure.
That shaded region has area (blank) square units.
After attempting this problem, I found the area to be
$A_1$(petal)-$A_2$(intersecting circle)$ = \displaystyle\int_0^\frac\pi 6 10\cos^2(3\theta)\,d\theta-\int_0^\frac\pi 9 (10\cos^2(3\theta)-5^2)\,d\theta$.
However, the website I'm using is telling me that my answer is incorrect.
Can someone tell me where I went wrong with this problem and what the correct answer is?
A big thank you in advance to anyone who is willing to help out.

When $x\in [\frac {\pi}{9},\frac {\pi}{6}]$ the curve for the "rose" is entirely inside the the circle, and the equation of the circle is ultimately irrelevant in calculating the area of the red region.
The shaded region for the first half petal is:
$\int_0^{\frac {\pi}{9}} \frac 12\cdot 5^2 + \int_{\frac {\pi}{9}}^{\frac {\pi}{6}} \frac 12 (10\cos 3\theta)^2 \ d\theta$
And remember when you find $r^2(\theta)$ you need to square the coefficients, too.