I am asked to calculate this integral and want to make sure I am doing this set up correctly, So I tried to paramatize this surface by :
$x = r\cos(\theta), \space y = r\sin(\theta), \space z = \sqrt{1 - r}$
$0 \leq r \leq 1 $ and $ 0 \leq \theta \leq 2\pi$
then I found :
$\Phi_{\theta} = -r\sin(\theta)i + r\cos(\theta)j + 0k , \space \Phi_{r} = \cos(\theta)i + \sin(\theta)j + \dfrac{1}{2\sqrt{1-r}}k$
which would make:
$\Phi_{\theta} \times \Phi_{r} = \dfrac{1}{2}\left(\dfrac{r\cos(\theta)}{\sqrt{1-r}}\right)i + \dfrac{1}{2}\left(\dfrac{r\sin(\theta)}{\sqrt{1-r}}\right)j -rk$
then my surface integral would be :
$$\int_0^{2\pi} \int_0^1 ((r\cos(\theta) + 3(r\sin(\theta))^5)i + (r\sin(\theta) + 10(r\cos(\theta))(\sqrt{1 - r}))j+$$ $$+ (\sqrt{1 - r} - (r\cos(\theta)r\sin(\theta))k )\space \cdot (\frac{1}{2}\left(\frac{r\cos(\theta)}{\sqrt{1-r}}\right)i + \frac{1}{2}\left(\frac{r\sin(\theta)}{\sqrt{1-r}}\right)j -rk )$$
is this set up correct ? I have tried to reduce it after taking the dot product with the trig identity's but I still end up with a fairly complicated expression
The problem is ambiguous. If the surface of the half ball includes the flat surface at $z=0$ the problem simplifies to a very simple integral using the divergence theorem. If not, the problem simplifies too but needs to calculate the flux through that flat surface.
1.- For the flat surface included ($B$ is the half ball, $\partial B$ its surface and $\mathbf F=F_x\mathbf i+F_y\mathbf j+F_z\mathbf k$), the divergence theorem gives:
$$\phi=\int_{\partial B}\mathbf F·\mathbb d\mathbf S=\int_B\nabla·\mathbf F\,\mathbb dV$$
$$\nabla·\mathbf F=\dfrac{\partial F_x}{\partial x}+\dfrac{\partial F_y}{\partial y}+\dfrac{\partial F_z}{\partial z}=1+1+1=3$$
In spherical coordinates $(r,\theta,\phi)$
$$\phi=\int_B 3\mathbb d V=\int_0^1\int_0^{\pi/2}\int_o^{2\pi}3r^2\sin\theta\,\mathbb d\phi\,\mathbb d\theta\,\mathbb d r=2\pi$$
2.- If the flat surface, $D=\{x^2+y^2\leq 1;z=0\}$, is not included we need to subtract the flux through it:
For the surface element with its vector pointing outwards is $-\mathbf k$:
$$\mathbb d\mathbf S=-\mathbb dx\mathbb dy\,\mathbf k$$
For the flux through the surface element ($z=0$):
$$\mathbf F·\mathbb d\mathbf S=((x + 3y^5)\mathbf i + (y + 10xz)\mathbf j + (z - xy)\mathbf k)·(-\mathbb dx\mathbb dy\,\mathbf k)=xy\,\mathbb dx\mathbb dy$$
$$\phi_D=\int_{-1}^1\int_{-\sqrt{1-x^2}}^{\sqrt{1-x^2}}xy\,\mathbb dx\mathbb dy=0$$
The flux is zero. It seems that in any case the answer is $2\pi$