I am studying for my linear algebra test and came across this lemma as an exercise:
"For any non-singular matrix $X$ and positive definite matrix $A$, $B = X^T A X$ is also positive definite."
Here I presume the dimensions of the matrix are assumed to be compatible.Since the exercise was just after the Cholesky Decomposition Theorem, I have tried using it for the solution.
Since $A$ is positive definite, $\exists!$ upper triangular matrix $R$ such that $A = R^T R$ by the Cholesky Decomposition Theorem. Now,
$B = X^TAX = X^T (R^TR)X = (RX)^T (RX)$
Assuming $A$ to be an $n \times n$ matrix and $X$ to be an $m \times n$ matrix, $B$ is an $m \times m $ matrix.
For a vector $v \in \mathbb{R}^M$,
$v^T B v = v^T(RX)^T (RX)v = (RXv)^T (RXv) = \langle RXv, RXv \rangle > 0$
Therefore, $B$ is positive definite. I guess I used the non-singularity of $X$ to show the inner product is not zero but I am not sure. Is this right?
In your argument, you've assumed that $RXv$ is nonzero, because the notion that $\langle w,w \rangle > 0$ is predicated on $w$ being nonzero.
A simpler approach is to say this:
We want to show that for any nonzero vector $u$, we have $u^tX^tA X u > 0$. Let $v = Xu$; then because $X$ is nonsingular, we know $v \ne 0$ (because "nonsingular" means "the kernel contains only the zero-vector"). So now we need to show that $v^t A v > 0$. But (because $v$ is nonzero) that follows from the definition of $A$ being positive definite.
===
Another approach is to say "Consider the quadratic form defined by $Q(v) = v^t A v$, where $v \in R^n$ is expressed in the standard basis. The matrix representation of this same quadratic form, in the basis consisting of the columns of $X$ (which form a basis because $X$ is nonsingular), is $X^t A X$. Because this is the same quadratic form, its values on nonzero vectors are still all positive."