I'm trying to solve the following ODE:
$$(1-x^2)\frac{d^2f}{dx^2}+\left(\frac{1}{x}-3x\right)\frac{df}{dx}+\left[\sigma-\frac{n^2}{1-x^2}-\frac{m^2}{x^2}\right]f=0$$
for $x \in [-1,1]$. We have, $n,m \in \mathbb{Z}$.
Wolfram alpha is able to give me a (very lengthy!) exact solution of this in terms of Hypergeometric functions, but I am more interested in the allowed values of the parameter $\sigma$.
Are there any arguments (along the lines of Sturm-Liouville theory, of which I know essentially nothing about) which constrain the allowed values of $\sigma$?
When solving laplaces equation in spherical polar coordinates, this kind of problem pops up in spherical harmonics. There, the parameter is constrained by requiring that the solution is regular at the regular singular points $x=\pm 1$ of the ODE. Can we do a similar thing here?
You can put your equation into Sturm-Liouville form on $(0,1)$ using the substitution $$ f=\frac{1}{\sqrt{x(1-x^2)}}g. $$ When you do that, the equation is transformed to $$ -\frac{d}{dx}\left((1-x^2)\frac{dg}{dx}\right)+\left(\frac{n^2}{1-x^2}+\frac{m^2-1/4}{x^2}\right)g=\left(\sigma-\frac{3}{4}\right)g. $$ This equation is formally selfadjoint in $L^2(0,1)$. My guess is that no endpoint conditions are required at $x=0$ or $x=1$ for $n \ge 0$, based on experience with the Associated Legendre equation, but I'm not really confident of that assessment at $x=0$. You can treat $\sigma-3/4$ as an eigenvalue parameter $\lambda$. It's not likely to be easy to determine the exact parameters $\lambda$ that will give you $L^2[0,1]$ eigenfunctions, if that's even what you get; it's always possible that the equation has only continuous spectrum. The SLEIGN2 code can probably help with this; it's free: http://www.math.niu.edu/SL2/