I need to use cylindrical coordinates to evaluate the volume of the sphere $$x^{2}+y^{2}+z^{2} = a^{2}$$ that's external to the ellipsoid $\frac{x^2}{a^2}+\frac{y^2}{a^2}+\frac{z^2}{b^2} = 1$.
So, $x$ and $y$ are equal for both, and i got $\frac{x^2}{a^2}+\frac{y^2}{a^2}+\frac{z^2}{a^2} = 1$ for the sphere but I don't know how to proceed.
What you need is to compute the following integral $$\int\int\int_{\Delta V}r dr d\theta dz$$ in cylindrical coordinates, where $\Delta V$ is your volume of integration. The first thing I notice is that $b<a$, otherwise the sphere is inside the ellipsoid. So now all we need is to find the limits of this integration volume. Due to symmetry in the $xy$ plane, the limits for the $\theta$ integration are from $0$ to $2\pi$. In principle $z$ has to be between $-a$ and $a$. Note however that, if you draw a cross section ($xz$ plane for example), you can identify regions along the $z$ axis where only the sphere is contributing, or you have both the sphere and the ellipse at the same coordinate. The ellipse extends from $-b$ to $b$.
Now for the limits of $r$: if $z>b$ or $z<-b$, you get $0\le r\le \sqrt{a^2-z^2}$. For the region between $-b$ and $b$, you need to be outside of the ellipse, so $\sqrt{a^2-z^2\frac{a^2}{b^2}}\le r \le \sqrt{a^2-z^2}$. This way you volume is $$2\pi\int_{-a}^{-b}dz\int_0^{\sqrt{a^2-z^2}}rdr+2\pi\int_{-b}^{b}dz\int_\sqrt{a^2-z^2\frac{a^2}{b^2}}^{\sqrt{a^2-z^2}}rdr+2\pi\int_{a}^{b}dz\int_0^{\sqrt{a^2-z^2}}rdr$$ Just for convenience, you can rewrite this as $$2\pi\int_{-a}^{a}dz\int_0^{\sqrt{a^2-z^2}}rdr-2\pi\int_{-b}^{b}dz\int_0^{\sqrt{a^2-z^2\frac{a^2}{b^2}}}rdr$$ That is to say volume of the sphere minus the volume of the ellipsoid. Your first integral in this form has to be $\frac{4}{3}\pi a^3$. The second integral is $\frac{4}{3}\pi a^2b$