Suppose $A$ is a $3\times 3$ symmetric matrix such that $$\begin{bmatrix}x & y &1\end{bmatrix} A \begin{bmatrix}x \\y & \\1 \end{bmatrix}=xy-1$$
How to prove $A$ has full rank and the number of positive eigenvalues of $A$ is one?
My try:
$$\begin{bmatrix}x & y &1\end{bmatrix} \begin{bmatrix}a & b &c \\b & d & e \\c & e & f \end{bmatrix} \begin{bmatrix}x \\y \\1 \end{bmatrix}=xy-1$$ $$\Rightarrow \begin{bmatrix}x & y &1\end{bmatrix} \begin{bmatrix}ax+by+c \\bx+dy+e \\cx+ey+f \end{bmatrix}=xy-1$$
$$\Rightarrow ax^2+2bxy+2cx+dy^2+2ey+f=xy-1 $$ $$\Rightarrow a=c=d=e=0\; \text{and} \;b=1/2, f=-1$$
So the matrix becomes
$$\begin{bmatrix}0 & 1/2 &0 \\1/2 & 0 & 0 \\0 & 0 & -1 \end{bmatrix}$$
which has only one positive eigenvalue, namely, $\frac 12$. Also this matrix is invertible and, hence, full rank.
Is this way correct or anything I'm doing wrong? What is the general idea?
Your argument is perfectly correct.
In general, you can convert a quadratic form $$ Ax^2 + 2B xy + C y^2 + 2Dx + 2Ey + F $$ into a matrix $$ M = \pmatrix{ A & B & D\\ B & C & E \\ D & E & F } $$ and the determinant of the matrix and trace can both be computed from the quadratic form, from which you can work out things like how many positive/negative eigenvalues there are.