I have solved the problem below. my answer is $2\pi a^3$ but in the text book the answer is $4\pi a^3$. Please I would appreciate if someone can tell me where i made a mistake.
PROBLEM
Find the Flux of F= $xi+yj+zk$ outward across the sphere $x^2+y^2+z^2=a^2$
SOLUTION Sphere: $$x^2+y^2+z^2=a^2$$ So $$z^2=a^2-x^2-y^2$$
$$\hat n dS = (i\frac{\partial z}{\partial x}+ j\frac{\partial z}{\partial y} +k)dA = \frac{x}{z}i+ \frac{y}{z}j +k)dA = \frac{1}{z}(xi+yj +zk)dA $$
$$F.\hat n dS = (xi+yj+zk). \frac{1}{z}(xi+yj +zk)dA = \frac{x^2+y^2+z^2}{z}dA= \frac{a^2}{\sqrt{a^2-x^2-y^2}}dA$$
$$F.\hat n dS = \frac{a^2}{\sqrt{a^2-r^2}}dA $$
I used the substitution u= $a^2 -r^2$
$$\iint_S F.\hat n\mathrm dS = \int_0^{2\pi} \int_0^{a} \frac{a^2}{\sqrt{a^2-r^2}}r\mathrm dr\mathrm d\theta = 2\pi a^3$$
You are considering the flux over an hemisphere, indeed we have
$$2\int_0^{2\pi} \int_0^{a} \frac{a^2}{\sqrt{a^2-r^2}}r\mathrm dr\mathrm d\theta = 4\pi\left[-a^2(a^2-r^2)^\frac12\right]_0^a =4\pi a^3$$
Note that $4\pi a^3$ is the correct result since the $|F|=a$ and $F$ is parallel to $n$ thus the flux is $4\pi a^2\cdot a$.