Let $S$ be the hemisphere with radius $1$ and centre along the $z$-axis in such a way that the south-pole of the hemisphere is located at the origin.
The mass function is $p(x,y,z) = 1-z$
I know that a sphere of radius 1 is given by the formula $x^2+y^2+z^2=1$. Knowing this I can solve for $z = \pm\sqrt{1-x^2-y^2}$, and then add +1 to lift the sphere up to $z=1$. $z= 1- \sqrt{1-x^2-y^2}$,
I can then insert for $z$ into the mass function. $$\iint 1-\left(1- \sqrt{1-x^2-y^2}\right) dA. $$
Further I can convert it into polar coordinates, $$\int_0^{2\pi}\int_0^{1}r\sqrt{1-r^2}dr d\theta.$$ I end up with the answer $\frac{2\pi}{3}$.
Is this correct?
It is not correct. Since at height $z$ we have a disc of radius $\sqrt{1-(1-z)^2}$ and constant density $(1-z)$, it follows, by integrating by sections, that the mass of the hemisphere is $$\int_{0}^1(1-z)\cdot \pi(1-(1-z)^2) dz=\frac{\pi}{4}.$$ The same result can be obtained following your approach: $$\int_{\theta=0}^{2\pi}\int_{r=0}^1\int_{z=1-\sqrt{1-r^2}}^{1}(1-z)\, (dz\, rdr\, d\theta).$$