Two cylindical shells of equal radius are inserted one into the other at various angle between the axes (I tried to give an example with the pic attached). What is the maximum volume for the cylindrical wedge of the upper cylinder into the bottom one? For which angle between the axes it is reached?
On
What makes this question challenging is that for two circular cylinders of equal radius, the points at which the "top" cylinder that coincide with the "rim" of the "bottom" cylinder are a function of the angle $\theta$ at which their bases intersect in a nontrivial way. As $\theta$ increases, the chord joining the two points of intersection at the base will become shorter until at $\theta = \pi/2$, it becomes a point and the cylinders no longer intersect.
Let us assume without loss of generality that the common radius is $1$. For a tilt angle of $\theta$ of the top cylinder, the boundary of horizontal slices are ellipses with semi-minor axis $1$ and semi-major axis $\sec \theta$, for $\theta \in [0, \pi/2)$. Thus we seek the nontrivial intersections of the ellipse $$\frac{(x - \sec \theta + 1)^2}{\sec^2 \theta} + y^2 = 1$$ with the unit circle $$x^2 + y^2 = 1.$$ This corresponds to $$x = -\tan^2 \frac{\theta}{2}, \quad y = \pm \sqrt{1 - \tan^4 \frac{\theta}{2}}. \tag{1}$$
The coordinate system in $(1)$ corresponds to the plane of the base (or "rim") of the bottom cylinder, with the origin at intersection of this plane with the cylinder's axis. So we need to transform these coordinates into the system corresponding to the desired integration region, which is the plane containing the base of the top cylinder and whose origin is the intersection of this plane with the top cylinder's axis. The projection of the points of intersection $(1)$ onto the base of the top cylinder have $x$-coordinate $$x = \cos \theta \sec \theta \tan^2 \frac{\theta}{2} = \tan^2 \frac{\theta}{2}.$$
Consequently, the desired volume is given by $$\begin{align} V(\theta) &= 2 \int_{x=\tan^2 \frac{\theta}{2}}^1 \int_{y=0}^{\sqrt{1-x^2}} \left(x - \tan^2 \frac{\theta}{2}\right) \tan \theta \, dy \, dx \\ &=2 \sec \theta \tan \frac{\theta}{2} \left( (\cos \theta - 1) \tan^{-1} \sqrt{\cos \theta} + \left(\tfrac{1}{8} \cos 2\theta + \tfrac{1}{6} \cos \theta + \tfrac{3}{8}\right) \sqrt{\cos \theta} \sec^4 \frac{\theta}{2} \right). \tag{2} \end{align}$$
The maximum volume corresponds to a critical point $\theta_\text{max}$ satisfying $0 = V'(\theta_\text{max})$. However, this does not appear to have an elementary closed-form solution. Numerical computation gives $$\theta_\text{max} \approx 0.94733192035127683061830353396077393584734024258456\ldots \tag{3}$$ radians, giving a maximum volume of $$V_\text{max} \approx 0.4485601341010714219395995688756010671999713477483\ldots \tag{4}$$
A plot of $V(\theta)$ is shown below:
Here is the configuration for the maximal $\theta$:
An animation of the two cylinders is shown below:
From the fact that the rim distance $d$ for both cylinders are the same, it can be established with the Pythagorean theorem $$(\frac d2)^2=r^2-(r-b)^2=r^2-(r-b\cos \theta)^2 $$ which leads to $b=\frac{2r}{1+\cos \theta}$. As a result, the height of the wedge is $$h= b\sin\theta =\frac{2r\sin \theta}{1+\cos \theta}$$ and the half angle $\phi$ extended from the rim center to the two rim-points is $$\cos \phi = \frac{r- b \cos \theta}r=\frac{1-\cos \theta}{1+\cos \theta}$$
Apply the volume formula of the wedge $$V(\theta)= \frac{hr^2}3\frac{3\sin\phi -3\phi\cos \phi-\sin^3\phi}{1-\cos \phi} $$ where both $h$ and $\phi$ are the functions of $\theta$ as derived previously. Unfortunately, due to the complexity of the volume expression, the analytical solution for the maximum is unlikely. The numerical evaluation reveals that the maximal volume is $V_{max}=0.4486r^3$, at $\theta_{max}=0.9473$, or approximately $54°$.