I want to define for which $a,b,c \in \mathbb{R}$ $$(X,Y) = X^T \begin{pmatrix} b+6 & 0 & 11 \\ 0 & 6 & a \\ 11 & -2 & 1 \end{pmatrix} Y + cy_1^2$$ is an inner product of $\mathbb{R}^3$. ($y_1$ first coord. of $Y$).
I have shown that $a = -2, c = 0$ by just controlling symmetry and bilinearity of $( \cdot, \cdot)$. For $b$, I am having trouble actually proving the boundaries of $b$. Using positive-definiteness, I have shown that $b > -6$ but I cannot prove that these are the only values of which $( \cdot, \cdot)$ is an inner product. How would I go about actually proving the boundaries of $b$?
Thank you.
Sylvester's criterion says the matrix is positive definite if and only if all its principal minors are positive. That is, calculate the determinant of the $1\times1$ matrix in the upper left corner, the $2\times2$ in the upper left corner, and the full $3\times3$; if those three numbers are all positive, you win.