For a usual KE+PE Schrodinger equation (which is just a second order ODE), I know that you cannot have a physically preparable discontinuous wavefunction solution. However I am interested in mathematially valid solutions for $\psi$.
I have heard that we can only guarantee that $\psi$ must be continuous if the potential function $V(x)$ is continuous. Does this mean that in the case of a finite square well $V(x)$ with a discontinuity, there is a valid discontinuous solution for $\psi$ (mathematically at least)?
$-\frac{\hbar^2}{2m} \frac{d^2\psi}{dx^2} + V(x) \psi = E \psi$
It dependson what you mean by "solution".
Typically, one would define a solution as a function $\psi:\mathbb{R}\to\mathbb{C}$ such that $-\frac{\hbar}{2m}\frac{d^2\psi}{dx^2}(x)+V(x)\psi(x)=E\psi(x)$ for all $x\in\mathbb{R}$. Clearly, for this to be the case, $\frac{d^2\psi}{dx^2}$ must be well defined, and so $\psi$ must be twice differentiable and thus continuous.
Often times, however, one is instead interested in weak solutions of the Schrödinger equation, which are solutions of a corresponding integral equation on some space of functions. Depending on the choice of weak formulation, weak solutions need not be functions at all (distributions or equivalence classes of functions are also common). Ultimately, whether discontinuous weak solutions exist (or if that statement even makes sense) depends on which weak formulation of the Schrödinger equation you're using.