I have a (homegrown) tool which takes a set of data points and finds the best-fit parabola by finding a rotation angle which gives the best vertical parabola by fitting the rotated data to a standard $ax^2 + bx + c$ form. My answer then can generate the putative parabola only by rotating by $\theta$ after generating the vertical parabola. That is, for $ y(x) = ax^2 + bx + c $ , the final curve is
$x' = x*cos(\theta) - y(x)*sin(\theta) $
$y' = x*sin(\theta) + y(x)*cos(\theta) $
What I'm looking for is an algorithm to convert the parameter set $ {(\theta,a,b,c)} $ to the form $ Ax^2 + Bxy + Cy^2 + Dx +Ey + F = 0$ .
Note: yes, I got into this mess by trying to write a Q&D fitting function rather than doing a nonlinear matrix minimization over $\{A,B,C,D,E,F\}$ .
There’s no need to solve for $y'$. In fact, for a rotated parabola, $y'$ is not a function of $x'$, anyway.
Recall that the inverse of a rotation through $\theta$ is a rotation through $-\theta$, so the inverse transformation is $$\begin{align}x&=x'\cos\theta+y'\sin\theta\\y&=-x'\sin\theta+y'\cos\theta.\end{align}$$ Substitute this into the equation $y=ax^2+bx+c$ and collect terms.
If this is too “ugly,” try writing the equations in matrix form. The general-form equation $Ax^2+Bxy+Cy^2+Dx+Ey+F=0$ can be expressed as $(x,y,1)\,M\,(x,y,1)^T=0$, where $$M=\pmatrix{A&\frac B2&\frac D2\\\frac B2&C&\frac E2\\\frac D2&\frac E2&F}.$$ Rotating the coordinate system amounts to replacing $(x,y,1)^T$ by $R\,(x',y',1)^T$, where $R$ is a homogeneous rotation matrix, so the coefficient matrix for the rotated quadratic will be $R^TMR$. If you multiply this out, you can then read the coefficients of the rotated general-form equation directly from the result.
In this particular case, we have $ax^2+bx-y+c-0$, so $$M=\pmatrix{a&0&\frac b2\\0&0&-\frac12\\\frac b2&-\frac12&c}.$$