I have a family of curves: $$F(x, y, t) = \sin(2t)x-\cos(2t)y-sin(t)$$
I'm trying to solve the equation for the envelope, that is this systems of equations: $$ \text{} \left\{ \begin{align} F(x, y, \theta) = \sin(2\theta)x - \cos(2\theta)y - \sin(\theta) = 0 \\ \frac{\partial F(x, y, \theta)}{\partial \theta} = 2x\cos(2\theta) - 2y\sin(2\theta) - \cos(\theta)\end{align} \right. $$
I know that I should end up with the following parametric equation: $$ (\sin(2t)\sin(t)+\frac{1}{2}\cos(2t)\cos(t), -\cos(2t)\sin(t)+\frac{1}{2}\sin(2t)\cos(t)) $$
But I don't know how to get there?
Repeating what you stated in your question, in matrix-vector format, we have
$ A r = b $
where
$ r = [x, y]^T $
$ A = \begin{bmatrix} \sin(2 \theta) && - \cos(2 \theta) \\ 2 \cos(2 \theta) && 2 \sin(2 \theta) \end{bmatrix}$
$ b = \begin{bmatrix} \sin \theta \\ \cos \theta \end{bmatrix} $
This linear system can be solved directly for $r$, using, for example, matrix inversion
$ r = A^{-1} b =\begin{bmatrix} \sin(2 \theta) && - \cos(2 \theta) \\ 2 \cos(2 \theta) && 2 \sin(2 \theta) \end{bmatrix}^{-1} \begin{bmatrix} \sin \theta \\ \cos \theta \end{bmatrix} = \frac{1}{2} \begin{bmatrix} 2 \sin(2 \theta) && \cos(2 \theta) \\ -2 \cos(2 \theta) && \sin(2 \theta) \end{bmatrix}\begin{bmatrix} \sin \theta \\ \cos \theta \end{bmatrix} \\ = \frac{1}{2} \begin{bmatrix} 2 \sin(2 \theta) \sin(\theta) + \cos(2 \theta) \cos(\theta) \\ - 2 \cos(2 \theta) \sin(\theta) + \sin(2 \theta) \cos(\theta) \end{bmatrix}$
Which is the expression that you have at the end of your question.