From [1, pag. 106] we learn that the set of all lines in in $\mathbb{R}^3$ is a $4$-$D$ manifold $\mathcal{S}$ embedded in $\mathbb{R}^6$:
Consider a space line $L$. Let $H$ be the point on $L$ closest to the coordinate origin $O$, and put $\boldsymbol{r}_H=\overrightarrow{O H}$. Let $\boldsymbol{m}$ be the unit vector that indicates the orientation of $L$. The space line $L$ is represented by two 3-vectors $\left\{\boldsymbol{m}, \boldsymbol{r}_H\right\}$; two representations $\left\{\boldsymbol{m}, \boldsymbol{r}_H\right\}$ and $\left\{-\boldsymbol{m}, \boldsymbol{r}_H\right\}$ define the same space line. A space point $\boldsymbol{r}$ is on $L$ if and only if vector $\boldsymbol{r}-\boldsymbol{r}_H$ is parallel to $\boldsymbol{m}$, or $$ \left(\boldsymbol{r}-\boldsymbol{r}_H\right) \times \boldsymbol{m}=\mathbf{0}, $$ which is simply the equation of space line $L$ if $\boldsymbol{r}$ is regarded as a variable. Two 3-vectors $\left\{\boldsymbol{m}, \boldsymbol{r}_H\right\}$ represent a space line if and only if $$ \|\boldsymbol{m}\|=1, \quad\left(\boldsymbol{m}, \boldsymbol{r}_H\right)=0, $$ where $(\cdot,\cdot)$ denotes the scalar product.
Now my question is: is $\mathcal{S}$ some well known manifold with a name? I am working on a problem where the position of a line in space has to be optimized and I would like to use manifold optimization techniques. I assume, for example, that $\mathcal{S}$ has a Lie Algebra, etc.
[1] Kanatani, K. (2005). Statistical Optimization for Geometric Computation. Dover Publications.
That is $\mathbb {RP}^2 \times \mathbb R^2$.
$\mathbb {RP}^2$ is for the line orientation noted $m$ (unit vector) in the OP. I first wrongly commented it was $S^2$, but we take the quotient by the equivalence relation $m \sim -m$. This is the definition of the real projective space, cf. https://en.wikipedia.org/wiki/Real_projective_space (where all non-null orientation vectors are considered, and quotient is by $m \sim \lambda m$ for all $\lambda \ne 0$).
$\mathbb {RP}^2$ is also the Grassmanian $\mathbf {Gr} (1, \mathbb R^3)$ (same Wikipedia page).
Then $\times \mathbb R^2$ is for the translation by $r_H$.
As $\mathbb {RP}^2$ can be embedded in $\mathbb R^4$ (see same Wikipedia page), $\mathbb {RP}^2 \times \mathbb R^2$ can be embedded in $\mathbb R^6$.