I am struggling with the following problem: Find the surface area of $x^2+y^2+z^2=a^2$ enclosed by the cylinder $\frac{x^2}{a^2}+ \frac{y^2}{b^2}=1$ $(a>b>0)$. The solution of the problem is supposed to be $4 \pi a^2 - 8a^2 \arcsin\left(\frac{\sqrt{a^2-b^2}}{a}\right)$.
I tried polar coordinates, I tried $x= a \, r \, cos(\theta)$, $y= b \, r \, sin(\theta)$ (with the Jacobian of this transformation equal to $abr$, and the limits of integration $0≤\theta ≤ \frac{\pi}{2}$ and $0≤ r ≤1$).
Time and time again, I get integranda which are impossible to integrate with respect to $\theta$ (and which also definitely won't give me an $\arcsin$ as a result).
Any help would be greatly appreciated.
It turns out we don't really need any calculus to solve this problem.
For completeness, let us first solve this problem the calculus way.
Method 1 - using spherical polar coordinates.
By symmetry, we only need to compute the area on the upper hemisphere and then multiply the result by $2$. Let $c^2 = a^2-b^2$ and parametrize the sphere by spherical polar coordinate
$$(x,y,z) = (a\sin\theta\cos\phi,a\sin\theta\sin\phi,a\cos\theta)$$
In the upper hemisphere, the portion of unit sphere within the ellipsoidal cylinder $$\left\{ (x,y,z) : \frac{x^2}{a^2} + \frac{y^2}{b^2} = 1 \right\}$$ is given by $$\frac{(a\sin\theta\cos\phi)^2}{a^2} + \frac{(a\sin\theta\sin\phi)^2}{b^2} \le 1 \iff \sin\theta \le \frac{1}{\sqrt{\cos^2\phi + \frac{a^2}{b^2}\sin^2\phi}} = \frac{1}{\sqrt{1 + \frac{c^2}{b^2}\sin^2\phi}} $$ This means as $\phi$ varies from $0$ to $2\pi$, $\theta$ can take values from $0$ to $\theta_0$ (dependent of $\phi$) where $\theta_0 \in [ 0, \frac{\pi}{2} ]$ is determined by $$\sin\theta_0 = \frac{1}{\sqrt{1+\frac{c^2}{b^2}\sin^2\phi}} \quad\iff\quad \cos\theta_0 = \frac{\frac{c}{b}|\sin\phi|}{\sqrt{1+\frac{c^2}{b^2}\sin^2\phi}} = \frac{\frac{c}{a}|\sin\phi|}{\sqrt{1-\frac{c^2}{a^2}\cos^2\phi}} $$ The area on the upper hemisphere we seek is then given by the integral
$$\begin{align} \verb/Area/_{up} &= a^2\int_0^{2\pi}\int_0^{\theta_0} \sin\theta d\theta d\phi = a^2\int_0^{2\pi}(1-\cos\theta_0) d\phi\\ &= a^2\left[ 2\pi - \int_0^{2\pi}\frac{\frac{c}{a}|\sin\phi|}{\sqrt{1-\frac{c^2}{a^2}\cos^2\phi}} d\phi \right] = a^2\left[ 2\pi - 4\int_0^{\pi/2} \frac{\frac{c}{a}\sin\phi}{\sqrt{1-\frac{c^2}{a^2}\cos^2\phi}} d\phi \right] \end{align} $$ Change variable to $t = \frac{c}{a}\cos\phi$, we get $$\verb/Area/_{up} = a^2\left[ 2\pi - 4\int_0^{c/a} \frac{dt}{\sqrt{1-t^2}} \right] = a^2\left[ 2\pi - 4\sin^{-1}\left(\frac{c}{a}\right)\right] $$ As a result, the area we seek is
$$\verb/Area/ = 2\verb/Area/_{up} = 4\pi a^2 - 8a^2\sin^{-1}\left(\frac{\sqrt{a^2-b^2}}{a}\right)$$
Method 2 - the geometric way.
As mentioned in beginning of this answer, we don't need any calculus to derive the result.
For any point $(x,y,z)$ on the intersection of the sphere and the ellipsoidal cylinder, we have
$$\frac{x^2}{a^2} + \frac{y^2}{a^2} + \frac{z^2}{a^2} = 1 = \frac{x^2}{a^2} + \frac{y^2}{b^2} \iff \frac{z^2}{a^2} = \frac{y^2}{b^2} - \frac{y^2}{a^2} = \frac{y^2c^2}{a^2b^2} \iff z = \pm \frac{c}{b} y $$ The RHS is the equation for a pair of planes.
What this means is the portion of upper hemisphere within the ellipsoidal cylinder is the portion of sphere above the two planes $z \ge \pm \frac{c}{b} y$. Since this two planes are making an angle $\tan^{-1}\left(\frac{c}{b}\right)$ with $xy$-plane, we find: $$\begin{align} \verb/Area/ = 2\verb/Area/_{up} &= 4a^2\left[\pi - 2\tan^{-1}\left(\frac{c}{b}\right)\right] = a^2\left[4\pi - 8\sin^{-1}\left(\frac{\frac{c}{b}}{\sqrt{1+\frac{c^2}{b^2}}}\right)\right]\\ &= a^2\left[4\pi - 8\sin^{-1}\left(\frac{c}{a}\right)\right] \end{align} $$ The same result we obtained by Method 1 above.