Also the center of the smaller one lies on the surface of the larger one.
This is what I have thus far.
Utilize the Cartesian plane we have for the smaller and larger spheres respectively: $(x-a)^2 + y^2 = b^2 $ and $x^2 + y^2 = a^2$
I solve for the $x$ value where these 2 intersect. I get $x_r = \frac{2a^2 - b^2}{2a}$
To get the volume of the larger cap in the intersection I used the method of cross sections. $\int^{a}_{x_r} \pi(a^2 - x^2)dx$
Similarly for the smaller sphere: $\int^{b}_{x_r} \pi(b^2 - (x-a)^2)dx$
Afterwards, I sum the result of the two to get the answer? Did I make a mistake somewhere
Your approach is fine. You are basically taking $x$ as your $z$ axis and hence in cylindrical coordinates $y^2 + z^2 = r^2$ and so you may want to write them as,
$r^2+x^2 = a^2$ and $r^2 + (x-a)^2 = b^2$.
Your first integral is fine but your second integral should be
$\displaystyle \int_{a-b}^{\frac{2a^2 - b^2}{2a}} \pi \, \big(b^2 - (a-x)^2\big) \,dx = \frac{\pi b^6}{24a^3} - \frac{\pi b^4}{2a} + \frac{2 \pi b^3}{3}$
The lower bound is the minimum value of $x$ for the smaller sphere, which is $(a-b)$ and it is part of the intersection volume.