I have a set of real-world images containing circles, but due to the images not being taken parallel to the circle plane, the circles appear as ellipses in the images. I need to crop out the inside of these circles accurately. Simply finding the intersections of the major and minor diameters with the ellipses does not result in points spaced 90° apart on the original circle, leading to inaccurate perspective distortion calculations.
I've come across research indicating that inferring the projection solely from the projected shape is not possible (refer to https://arxiv.org/abs/1902.04541). I'm seeking guidance on adding additional information to enable the reversal of perspective distortion. While marking four points on the original circle that are 90° apart would be the easiest solution, I'm curious if there are other viable approaches. Would two opposite points or the projected position center point be sufficient? Additionally, could knowing the image center, assuming a regular camera, be beneficial? It's worth mentioning that I am currently disregarding lens distortion.
Here's an example image showcasing the issue: example
I would appreciate any insights or suggestions on how to address this challenge effectively.