So in computer graphics there is a term called the plenoptic function. This function also arises in computer vision and apparently biomedical imaging. Mathematically it is usually just described as a $5D$ function $L(x, y, z,$ $\theta$, $\phi$), where $L$ is radiance, $(x, y, z)$ is a coordinate in $R^3$ and $(\theta, \phi)$ are spherical coordinates for an outgoing ray direction. But in free space (i.e. space empty of matter) radiance is conserved along rays. So suppose we have some free space region, then in computer graphics we might reduce the $5D$ function to a lower dimensionality by considering equivalent some of these rays. For example, we could place two planes around a volume of free space, and parameterize one plane by $(x, y)$ and the other by $(u, v)$, and then describe the radiance in the free space as $L(x, y, u, v)$.
In topology this "considering equivalent" would correspond to making a quotient space under an equivalence relation. Now if all the rays happened to pass through the origin this would correspond to the real projective space, which is a well-studied topological space. But in the case of the plenoptic function the rays do not in general pass through the origin. I guess one way to construct the topological space that identifies rays together as in the plenoptic function for $3D$ Euclidean space would be to start with the "space of all rays" $R^3 \times RP^2$ and identify points $(p_1, v_1)$ and $(p_2, v_2)$ if $v_1 \sim v_2$ and $p_2 - p_1 \sim v_1$, where ~ indicates the equivalence relation for $RP^2$. For geometric intuition, this could be explained as the ray directions ($v_1, v_2)$ leaving the points $(p_1, p_2)$ (respectively) are the same and the difference between the points is consistent with the ray direction. I guess this could also be generalized to $R^n \times RP^{n-1}$ in the same way. And technically since radiance is only conserved in free space we usually would be looking at subspaces of a space thus constructed. In any case, my question boils down to: does this "ray projective" topological space have any name in math, is it well-studied and understood, and are there any references that might be suggested for it? Thanks!
I was going to write up a detailed answer, but judging by your comment it sounds like you know what's going on. Feel free to comment on this answer with anything you want me to elaborate on ^_^
The short answer is that the (affine) lines in $\mathbb{R}^3$ are parametrized by points in $\text{Graff}(3,1)$ the Affine Grassmannians. You can read a detailed account of this object (as well as find references to other relevant facts) in "The Grassmannian of Affine Subspaces" by Lim, Wong, and Ye. It is available on the arxiv here.
I hope this helps ^_^