I have to find $$\iint \limits_{S} \vec{F}\cdot \vec{N} dA$$ where $$\vec{F} = (2x^2,\frac{y^2}{2},-cos\pi z)$$ and S is the surface of tetrahedron of vertices $$(0,0,0),(1,0,0),(0,1,0),(0,0,1)$$
we know that from Gauss Divergence theorem that,
$$\iint\limits_{S} \vec{F} \cdot \vec{N} dA = \iiint\limits_{V} div \vec{F} dV$$
The divergence of the given Function is,
$$div \vec{F} = 4x + y + \pi sin \pi z$$
so $$\iint\limits_{S} \vec{F} \cdot \vec{N} dA = \iiint\limits_{V} 4x + y + \pi sin \pi z dV$$
But I am stuck finding the limit of integration.How to find the limits? What procedure should I follow.
I am really struggling to find the limit of integration.
As you mentioned, the surface integral using divergence theorem is
$ \displaystyle \iiint_{E} (4x + y + \pi \sin \pi z) ~dV$
where region $E$ is interior of the tetrahedron. The tetrahedron is bound by x, y and z coordinate planes, and the plane $x + y +z = 1$
The projection of the region in xy-plane is $~x + y \leq 1, x, y \ge 0$.
So the integral will be,
$ \displaystyle \int_0^1 \int_0^{1-x} \int_0^{1- x - y} (4x + y + \pi \sin \pi z) ~ dz ~ dy ~ dx$
You can write the integral in any order but for $ \sin \pi z$ component, integrating wrt $~dz$ first may be a bit easier to evaluate .