I have been relearning quantum mechanics recently since I realized I got stuck in some misunderstanding of fundamentals when trying to solve certain problems. My question is actually simply the following:
What is the relationship between the measurement postulate, stated in terms of projection valued measures, and the eigenvectors of observables in the case of an infinite dimensional Hilbert space?
Now here is some context to maybe see where my confusion comes from:
Most QM textbooks advertise that observables are postulated to be self-adjoint operators because the spectral theorem assures you that their eigenvectors form a basis, which is connected to the measurement postulate via the idea that when you expand the state of the system in the eigenvectors of an observable, the coefficients of this expansion are the probability amplitudes for the measurement of said observable. I was stuck with the idea that somehow you can always use the eigenvectors of some observable to decompose any state. However, now I know that this is only true for finite dimensional Hilbert spaces.
On infinite dimensional Hilbert spaces, the eigenvectors of self-adjoint operators need not even be in the Hilbert space (for example, the momentum operator $\hat{p}:D(\hat{p})\rightarrow L^2(\mathbb{R})$, $\hat{p}=-i\partial_x$, which has a self-adjoint extention, has eigenvectors $\psi_p(x) = e^{ipx}$, which aren't square integrable functions. Even so, while they do not really form a basis in the Hilbert space, they still play an important role (related to Fourier transforms).
Now, measurement, formulated in a more rigorous manner, has to do with the actual spectral theorem, namely that for any self-adjoint operator there exists a projection-valued measure $P_A$ such that $A$ can be represented in terms of the Lebesgue-Stieltjes integral as $$A = \int_\mathbb{R} \lambda dP_A(\lambda)$$
Now from my understanding so far, PVMs on finite dimensional spaces and the projectors described in QM texbooks as say $P_\psi = |\psi><\psi|$ in Dirac notation (which again is a notation enabled by working with bases given by the eigenvectors of observables) are actually two sides of the same coin so to say. But I cannot seem to wrap my head around (or find any resource on the internet) about the infinite dimensional case.
Here would be some of the questions I'm pondering regarding this:
What is the connection between PVM and eigenvectors in the case of the momentum operator for example?
Are eigenvectors of operators actually relevant in the infinite dimensional case in general? (or the momentum/position are some very special cases?)
If the answer to 2 is affirmative, what happens to operators whose eigenvectors have discontinuities with infinite jumps? (one such an example I can think of is: $\hat{T}:D(T)\rightarrow \{f \in L^2(\left[0,2\pi\right])|f(0)=f(2\pi)\}$, $\hat{T} = i(cos^2 (\theta) - 1/2)\frac{d}{d \theta} + i\, sin(\theta) cos(\theta)$, for which there should be a self-adjoint extension, or rather some $D(T)$ for which it is self-adjoint - if I'm wrong scream at me, please. The primitive $\int \frac{sin(\theta)cos(\theta)}{cos^2(\theta)-1/2} d\theta$ diverges to $+\infty$ at the points $\theta_0\in\{\pi/4,\, 3\pi/4,\, 5\pi/4,\, 7\pi/4\}$ ).
How would one treat the measurement of the operator described in 3.?