I am looking at
$$A(x − h)^2 + B(x − h)(y − k) + C(y − k)^2 = 1$$
This is a rotating ellipse formula, where $h,k$ are the centroid of the ellipse. I have tried looking around for $A,B,C$ parameters, and I see that they are from Quadratic formula. But to be frank, I want to see how $A,B,C$ impact the orientation of the ellipse and I am looking for a more visual explanation for this question.
On
The axes of the ellipse are in the directions of the eigenvectors of the $2 \times 2$ symmetric matrix $$M = \pmatrix{A & B/2\cr B/2 & C\cr}$$ which corresponds to the quadratic form: $$ A x^2 + B x y + C y^2 = (x\ y) M \pmatrix{x\cr y\cr}$$ The semi-major and semi-minor axes are then $\lambda^{-1/2}$ where $\lambda$ are the eigenvalues.
Thus suppose you want an ellipse with centroid at the origin (any other centroid, of course, can be obtained by translation), semi-axes $a$ and $b$ with the major axis at angle $\theta$ counterclockwise from the positive $x$ axis. The eigenvectors can be taken to be $\pmatrix{\cos \theta\cr \sin \theta\cr}$ and $\pmatrix{-\sin \theta\cr \cos \theta\cr}$ for eigenvalues $1/a^2$ and $1/b^2$ respectively, and then $$M = \pmatrix{\cos \theta & -\sin \theta\cr \sin \theta & \cos \theta\cr} \pmatrix{1/a^2 & 0\cr 0 & 1/b^2\cr} \pmatrix{\cos \theta & \sin \theta\cr -\sin \theta & \cos \theta\cr} $$ We get $A = \dfrac{\cos^2 \theta}{a^2} + \dfrac{\sin^2 \theta}{b^2}$, $B = \left(\dfrac{1}{a^2} - \dfrac{1}{b^2}\right) \sin(2\theta)$, $C = \dfrac{\sin^2 \theta}{a^2} + \dfrac{\cos^2 \theta}{b^2}$.
On
I would like to explain the transforms on a picture.
$$\frac{x'^2}{a^2}+\frac{y'^2}{b^2}=1$$
$$x=x'.\cos \alpha - y'. \sin \alpha $$
$$y=x'.\sin \alpha + y'. \cos \alpha $$
We can get easily the result above from the picture
$$\cos \alpha. x=\cos \alpha (x'.\cos \alpha - y'. \sin \alpha) $$
$$\sin \alpha. y=\sin \alpha (x'.\sin \alpha + y'. \cos \alpha) $$
$$x'=x.\cos \alpha + y. \sin \alpha $$
$$-\sin \alpha. x=\cos \alpha (x'.\cos \alpha - y'. \sin \alpha) $$
$$\cos \alpha. y=\sin \alpha (x'.\sin \alpha + y'. \cos \alpha) $$
$$y'=-x.\sin \alpha + y. \cos \alpha $$
$$\frac{(x.\cos \alpha + y. \sin \alpha)^2}{a^2}+\frac{(-x.\sin \alpha + y. \cos \alpha)^2}{b^2}=1$$
$$(\frac{\cos^2 \alpha }{a^2}+\frac{\sin^2 \alpha }{b^2})x^2+2(\frac{\cos \alpha \sin \alpha }{a^2}-\frac{\cos \alpha \sin \alpha }{b^2})xy+(\frac{\sin^2 \alpha }{a^2}+\frac{\cos^2 \alpha }{b^2})y^2=1$$
$$(\frac{\cos^2 \alpha }{a^2}+\frac{\sin^2 \alpha }{b^2})x^2+ \sin 2\alpha (\frac{1 }{a^2}-\frac{1}{b^2})xy+(\frac{\sin^2 \alpha }{a^2}+\frac{\cos^2 \alpha }{b^2})y^2=1$$
$$Ax^2+ Bxy+Cy^2=1$$
$x=X-h$
$y=Y-k$
$$A(X-h)^2+ B(X-h)(Y-k)+C(Y-k)^2=1$$
$$A=\frac{\cos^2 \alpha }{a^2}+\frac{\sin^2 \alpha }{b^2}$$
$$B=\sin 2\alpha (\frac{1 }{a^2}-\frac{1}{b^2})$$
$$C=\frac{\sin^2 \alpha }{a^2}+\frac{\cos^2 \alpha }{b^2}$$
I interpreted the question as asking for intuition about the parameters directly. So here is an alternative explanation. Of course, for doing anything useful with the ellipse, the eigenvalue/eigenvector perspective in Robert's answer is usually the most valuable, as it is coordinate-independent and more mathematically natural.
The ellipse is centered at the point $(h,k)$. It passes through the four points $(h\pm a,k)$ and $(h,k\pm c)$, where $a = 1/\sqrt A$ and $c = 1/\sqrt C$. That is, $A$ and $C$ tell you where the ellipse meets the horizontal and vertical lines through its center.
It's hard to say anything quantitative about the effect of $B$ without involving eigenvalues and eigenvectors, as in Robert's answer. Qualitatively, it controls how the ellipse deviates from being axis-aligned.
Above are three ellipses, each with $A=1/4$ and $C=1$, and with varying values of $B$. As you can see, all of them have the same span along the horizontal axis, $a = 1/\sqrt{1/4} = 2$, and similarly in the vertical, $c = 1/\sqrt 1 = 1$. When $B = 0$ (blue), the ellipse is axis-aligned. When it is positive ($B = 1/2$, maroon), the ellipse is oriented like a "\"; when it is negative ($B = -1/2$, yellow), it is oriented like a "/".