To calculate the moment of inertia of arbitrary shape,
you have to integrate $dI$ for both $r$ and $\theta$
Basically $dI=dm r^2$ , $\int_0^R \int_0^\pi dr d\theta$ in polar coordinates.
If the total mass is M, then $dm=M\frac{(partial-area)}{(total-area)}$
I tried to integrate the following picture in the polar coordinate, only the light-blue colored part that looks like a fidget spinner.
but the integration was too hard.
Please help!
Integration over the whole domain is not the easier option nor the recommended one. The body is a collection of six circles, so the momentum of inertia should be calculated as a sum of six (in this case equal) terms. Let $r$ be the radius of the circle and $m=M/6$ the mass of a single circle. The momentum of inertia with respect to the center of the circle is $$ I_1=\frac12 m r^2 $$ By the Huygens-Steiner theorem the momentum with respect to the origin $O$ is $$ I_2=\frac12 m r^2+md^2 $$ where $d$ is the distance between the origin and the center of the circle. From the figure, it is apparent that $d=2r$, then $$ I_2=\frac92 m r^2 $$ Therefore $$ I=6 I_2=27mr^2=\frac92 M r^2 $$