Consider a quadratic form $q(x)=x^TAx$ where $A$ is a real $n\times n$ symmetric matrix. If we diagonalize $A$, using a nonorthogonal matrix $P$, i.e. $A=PDP^{-1}$ where $D$ is a diagonal matrix of eigenvalues, then does the quadratic equation $x^TDx+c=0$ represent the same geometric shape as $x^TAx+c=0$? In other words, does a nonorthogonal change of coordinates necessarily not alter the basic shape? So does an ellipse remains an ellipse albeit it becomes distorted? If so, my question is why?
I think this is because the coordinate transformation is invertible but I cannot formally explain how that helps.
The short answer is no, the quadric’s basic shape might not be preserved.
The overall shape of the quadric (hyper)surface defined by the implicit equation $q(\mathbf x)=\mathbf x^TA\mathbf x=1$ is determined by its spectrum: the number of positive, negative and zero eigenvalues of the symmetric matrix $A$. In the plane, $++$ and $--$ correspond to ellipses, $+-$ to parabolas, $+0$ and $-0$ to parabolas, and $00$ to various degenerate conics. Note that for the nondegenerate conics, the spectrum is completely determined by the sign of $\det(A)$.
If you have a linear transformation represented by the matrix $M$, so that $\mathbf x'=M\mathbf x$, then $$\mathbf x^TA\mathbf x = (M^{-1}\mathbf x)^TA(M^{-1}\mathbf x) = \mathbf x^T(M^{-T}AM^{-1})\mathbf x.$$ What this means is that, when interpreting $A$ as the matrix of a quadratic form, a change of basis takes the form $P^TAP$, not $P^{-1}AP$ as it does when $A$ is the matrix of a linear transformation. This is the correct expression to use for a quadratic form whether or not $P$ is orthogonal. The Spectral Theorem assures us that such linear transformations don’t change the overall shape of the quadric. So, in particular, the image of an ellipse under an invertible linear transformation is another ellipse, and so on.
On the other hand, the expression that you’re asking about, $P^{-1}AP$, doesn’t represent a linear change of coordinates of the quadratic form (unless, of course, $P$ is orthogonal). Now, $\det(A)=\det(P^{-1}AP)$, so one might think that at least in the two-dimensional case the overall shape of the two conics is the same. Unfortunately, we’re not really interested in the eigenvalues of $B=P^{-1}AP$: only the symmetric part of a matrix contributes to the quadratic form, so we’re really interested in the spectrum of $B+B^T$. In general, though, there’s no systematic relationship between the eigenvalues of a matrix and its symmetric part. Indeed, it’s not too hard to construct a $2\times2$ example for which the the signs of $\det(B)$ and $\det(B+B^T)$ differ.
For example, let $$B=\begin{bmatrix}2&3+\sqrt6\\3-\sqrt6&2\end{bmatrix}.$$ Then $\det(B)=1$, but the determinant of its symmetric part is $-5$. We have $$B = \begin{bmatrix}-\sqrt3 & \sqrt3 \\ 3-\sqrt6 & 3-\sqrt6\end{bmatrix} \begin{bmatrix}2-\sqrt3&0\\0&2+\sqrt3\end{bmatrix} \begin{bmatrix}-\sqrt3 & \sqrt3 \\ 3-\sqrt6 & 3-\sqrt6\end{bmatrix}^{-1},$$ so taking $D$ to be the diagonal matrix in the middle and $P$ the matrix on the left, $\mathbf x^TD\mathbf x=1$ is the ellipse $(2-\sqrt3)x^2+(2+\sqrt3)y^2=1$, but $\mathbf x^T(PDP^{-1})\mathbf x=1$ is the hyperbola $2x^2+6xy+2y^2=1$. At the risk of belaboring the obvious, this demonstrates that if you have a diagonal matrix $D$ and invertible $P$, then the basic shapes of the curves $\mathbf x^TD\mathbf x=1$ and $\mathbf x^T(PDP^{-1})\mathbf x=1$ aren’t necessarily the same.
Actually, we can do a bit better that this in the $2\times2$ case. Consider the symmetric matrix $$Q = \begin{bmatrix}a&b\\b&d\end{bmatrix}$$ and let $$M = Q+\begin{bmatrix}0&c\\-c&0\end{bmatrix}.$$ Then $\det(Q)=ad-b^2$ and $\det(M)=ad-b^2+c^2\ge\det(Q)$. So in particular, if $\det(Q)\gt0$ then $\det(M)\gt0$, so if you end up with an ellipse via $M=PDP^{-1}$, then you must also have started with an ellipse. I don’t know if this holds in higher dimensions, but it seems an interesting problem in its own right.