Assume that I have a 3D circle with a center at $(c_1, c_2, c_3)$ in the circle coordinate frame $C$. The radius of the circle is $r$, and there is a unit vector $(v_1, v_2, v_3)$ (also in coordinate frame $C$) normal to the plane of the circle at the center point.
I have a pinhole camera located at point $(k_1, k_2, k_3)$ in the camera coordinate frame $K$. I have a known camera-to-circle transformation matrix $T_{C}^{K}$ that transforms any point in $C$ to coordinate frame $K$. The camera has a known intrinsic camera matrix $N$.
How do I project the 3D circle onto the image plane of the camera in order to find the parameters of the resulting ellipse / line segment / curve in the image plane.
Coordinate frames:
Let $\vec{p}_{c}=(x_c, \ y_c, \ z_c)$ be a point in the coordinate frame $C$ and $\vec{p}_{k}=(x_k, \ y_k, \ z_k)$ the same point in the coordinate frame $K$. The way we transform one point to another is by a $3 \times 3$ matrix $T_{ck}$ such
$$\vec{p}_{k} = T_{ck} \cdot \vec{p}_{c} + \vec{s}_{k}$$
The vector $\vec{s}_{k}$ is position of the origin $O_{c}$ on $K$.
The same happens for the camera: There's a transformation matrix $T_{kn}$ to transform a point $\vec{p}_{k} \in K$ to the point $\vec{p}_{n} \in N$
$$\vec{p}_{n} = T_{kn} \cdot \vec{p}_{k} + \vec{h}_{n}$$
For the camera, the origin $O_{k}$ is represented at the position $\vec{h}_{n}$.
Therefore, transforming any point $\vec{p}_{c}$ to a point $\vec{p}_{n}$ is made by
$$\vec{p}_{n} = \vec{h}_{n} + T_{kn} \cdot \vec{s}_{k} + T_{kn} T_{ck} \cdot \vec{p}_{c}$$
Lose information:
In the coordinate frame $K$, there's a point $\vec{k}_{k} = \left(k_1, \ k_2, \ k_3\right)$, which we call the pin-hole. The pin-hole camera always loses information. Let $\vec{n}_{k}$ be a unit vector perpendicular to the plane of the camera.
All the points that lie on the line $\vec{p}_{k} = \vec{k}_{k} + t \cdot \vec{n}_{k}$ are the same for the camera. So
$$\begin{align}\vec{p}_{n} & = \vec{h}_{n} + T_{kn} \cdot \vec{p}_{k} \\ & = \vec{h}_{n} + T_{kn} \cdot \left(\vec{k}_{k} + t \cdot \vec{n}_{k} \right) \\ & = \vec{h}_{n} + T_{kn} \cdot \vec{k}_{k} + t \cdot T_{kn} \cdot \vec{n}_{k}\end{align}$$
$$\boxed{T_{kn} \cdot \vec{n}_{k} = \vec{0}}$$
Normally that happens cause $T_{nk}$ normally is a transformation from $\mathbb{R}^{3}$ into $\mathbb{R}^{2}$:
$$T_{nk} = \begin{bmatrix}\vec{a}_{k} \\ \vec{b}_{k} \end{bmatrix} = \begin{bmatrix}a_1 & a_2 & a_ 3 \\ b_1 & b_2 & b_3 \end{bmatrix}$$ $$\langle \vec{a}_{k}, \vec{n}_{k}\rangle = 0 \ \ \ \ \ \ \text{and} \ \ \ \ \ \ \ \ \langle \vec{b}_{k}, \vec{n}_{k}\rangle = 0$$
Therefore, we can transform a point $\vec{p}_{k}$ into $\vec{q}_{k}$ by taking out the projection to the line at first.
$$\vec{q}_{k} = \underbrace{\dfrac{\langle \vec{a}_{k}, \ \vec{p}_{k}\rangle}{\langle \vec{a}_k, \ \vec{a}_{k} \rangle }}_{\alpha} \cdot \vec{a}_{k} + \underbrace{\dfrac{\langle \vec{b}_{k}, \ \vec{p}_{k}\rangle}{\langle \vec{b}_k, \ \vec{b}_{k} \rangle }}_{\beta} \cdot \vec{b}_{k}$$
And for $\vec{q}_{n}$ we get
$$\vec{q}_{n} = \vec{h}_{n} + T_{kn} \cdot \left(\alpha \cdot \vec{a}_{k} + \beta \cdot \vec{b}_{k}\right)$$
$$\begin{bmatrix}q_{n1} \\ q_{n2}\end{bmatrix} = \begin{bmatrix}h_{n1} \\ h_{n2}\end{bmatrix} + \underbrace{\begin{bmatrix}1 & \dfrac{\langle \vec{b}_{k}, \ \vec{a}_{k}\rangle}{\langle \vec{b}_k, \ \vec{b}_{k} \rangle } \\ \dfrac{\langle \vec{a}_{k}, \ \vec{b}_{k}\rangle}{\langle \vec{a}_k, \ \vec{a}_{k} \rangle } & 1\end{bmatrix}}_{H_k}\begin{bmatrix}\langle \vec{a}_{k}, \ \vec{p}_{k}\rangle\\ \langle \vec{b}_{k}, \ \vec{p}_{k}\rangle\end{bmatrix}$$
The circle:
There's a circle which center is given by $\vec{c}_{c} = (c_1, \ c_2, \ c_3)$ and has radius $r$. Its perpendicular vector (to the plane which contains the circle) is $\vec{v}_{c} = (v_1, \ v_2, \ v_3)$.
Let $\vec{g}_c$ be a point in the circle, then
$$\|\vec{g}_{c} - \vec{c}_{c}\| = r\label{1}\tag{1}$$ $$\left(\vec{g}_{c} - \vec{c}_{c}\right) \times \vec{v}_{c} = \vec{0}\label{2}\tag{2}$$
Now, let's call $\vec{u}_c$ and $\vec{w}_c$ two unit vectors that are parallel to the plane of the circle, such
$$\vec{u}_c \perp \vec{w}_c \perp \vec{v}_c \perp \vec{u}_c$$ $$\|\vec{u}_c\| = 1 \ \ \ \ \ \ \ \ \ \|\vec{w}_c\| = 1$$
From \eqref{1} and \eqref{2}
$$\vec{g}_{c} = \vec{c}_c + \vec{u}_{c} \cdot r\cos \theta + \vec{w}_c \cdot r\sin \theta$$
Transforming to $N$:
$$\begin{align}\vec{g}_{n} & = \vec{h}_{n} + T_{kn} \cdot \vec{s}_{k} + T_{kn} T_{ck} \cdot \vec{g}_{c} \\ & = \vec{h}_{n} + T_{kn} \cdot \vec{s}_{k} + T_{kn} T_{ck} \cdot \vec{c}_{c} + r \cos \theta \cdot T_{kn} T_{ck} \cdot \vec{u}_{c} + r \sin \theta \cdot T_{kn} T_{ck} \cdot \vec{w}_{c}\end{align}$$
Renaming
$$\vec{g}_{n} = \vec{f}_{n} + r \cos \theta \cdot \vec{u}_{n} + r \sin \theta \cdot \vec{w}_{n}$$
With
$$\begin{align*}\vec{f}_{n} & = \vec{h}_{n} + T_{kn} \cdot \left( \vec{s}_{k} + T_{ck} \cdot \vec{c}_{c}\right) \\ \vec{u}_{n} & = T_{kn} T_{ck} \cdot \vec{u}_{c} \\ \vec{w}_{n} & = T_{kn} T_{ck} \cdot \vec{w}_{c} \end{align*}$$
Then you will get an ellipse of center $\vec{f}_n$ and axis $\vec{u}_n$ and $\vec{w}_{n}$. If one of this two vectors are zero, then you will get a line.
There's a python code below to represent a projection from a 3D space to a 2D space (of camera) using the coordinates. The symbol
@represents matrix multiplication.