Positive-Definiteness of a Quadratic Form Matrix

Question

I'm having trouble with some maths regarding the expression of the matrix quadratic form (i.e. $x^TAx$) and the proof that, where the eigenvalues $\lambda_1,\lambda_2,...,\lambda_n$ are all positive, the quadratic form is positive definite.

My understanding is that the definition of positive-definiteness is when $x^TAx>0$ for all x where at least one element of $x \neq 0$.

My textbook produces the following proof, but I don't understand the last line:

Where $s_i$ = a normalized eigenvector of A, $\lambda_i$ = the corresponding eigenvalue, and $C = [s_1|s_2|...|s_n]$, $C^{-1}AC=D=$ the "diagonalization" of A.

Since the eigenvectors of A are orthonormal, $C^{-1} = C^T$.

Suppose we define a transformation $x=CX$. Then the equation becomes: $x^TAx = (CX)^{T}ACX = X^TDX = \lambda_1X_1^2 + \lambda_2X_2^2+...+\lambda_nX_n^2$.

It follows from this that a quadratic form is positive-definite if and only if all its eigenvalues are positive.

So, in summary, I don't understand why the following derivation true:

$x^TAx = \lambda_1X_1^2 + \lambda_2X_2^2+...+\lambda_nX_n^2$ therefore, for $x^TAx > 0$ for all x where at least one element of $x_i \neq 0$ to be true, $\lambda_1, \lambda_2, ..., \lambda_n > 0$.

Can someone please help with the derivation of the last step?a

Thanks in advance for your help.

Answer 1

I misread the question. @WillOrrick's comment is relevant, if $A$ is positive definite, then you must have $x^T Ax > 0$ for any $x \neq 0$ (which is equivalent to at least one $x_i \neq 0$).

To illustrate why the 'for any' is important, consider the matrix $B = \left( \begin{array}{cc} 1 & 0 \\ 0 & -1 \end{array} \right)$. Then $e_1^T B e_1 > 0$, but $B$ is not positive definite. The 'for any $x \neq 0$' is important.

Since you have $x^TAx = \sum_{i=1}^n \lambda_i X_i^2$ (with $x = CX$), we may choose $x=C e_k$, where $e_k$ is the $k$th basis vector (ie, $x$ is an eigenvector corresponding to $\lambda_k$). It follows that for this choice of $x$, we have $x^TAx = \sum_{i=1}^n \lambda_i X_i = \lambda_k X_k = \lambda_k$. Since $A$ is positive definite, it follows that $\lambda_k>0$. Since $k$ was arbitrary, we have that all eigenvalues are strictly positive.

Answer 2

For the "if" part: if all of the $\lambda_j$ are positive then $\lambda_1X_1^2+\ldots+\lambda_nX_n^2$ is positive for all non-zero vectors $X$ and hence for all non-zero vectors $x$.

For the "only if" part: Suppose that $\lambda_1X_1^2+\ldots+\lambda_nX_n^2$ is positive for all non-zero vectors $x$. Then in particular, it is positive for the vector $x$ corresponding to the $X$ with $X_j=1$ and $X_k=0$ for all $k\ne j$. Therefore $\lambda_j$ is positive.

Answer 3

The "only if" part (if $A$ is pos.def, then all eigenvalues are positive) can be proved more easily by absurd. Suppose that $x^t A x>0$ for all $x\ne 0$, but we have some $\lambda_i <0 $. The, calling $p_i$ ($\ne 0$) the associated eigenvector, we have $A p_i = \lambda_i p_i$ and multipliyin by $p_i^t$ we get $ p_i^t A p_i = \lambda_i |p_i|^2 <0 $ which contradicts the initial assumption.