What does the following mean? I know that $x^Tx$ is the magnitude of $x$. What does the following formula represent intuitively? $x$ is a vector and $A$ is some scaling matrix. The given is $x^T A x i$
(Ignore the i)
I'm learning about positive definiteness.
On
If $X$ is a vector in $n$ dimensional space, it is an $n \times 1$ matrix-a column. $X^T$ is a $1 \times n$ matrix-a row. $A$ is an $n \times n$ matrix, which you might see as the matrix of a linear transformation on your space. $AX$ is the vector that $A$ transforms $X$ into. It is again $n \times 1$. Then $X^TAX$ is a $1 \times 1$ matrix, a number. $A$ is positive definite if the number is positive for all nonzero $X$. An example is the identity matrix, because $IX=X$ and we know $X^TX$ is positive for all nonzero $X$.
On
Rather than thinking about $y^T x$, it's better to think about the dot product $y \cdot x$. Recall that in a real vector space, the dot product $y \cdot x$ gives you a measure of how the vectors $x$ and $y$ project onto each other: if the dot product is large and positive, the vectors $x$ and $y$ are pointing in a similar direction, and if it is large and negative, the vectors are pointing in an opposite direction.
Since $x^T A x = x \cdot (Ax)$, it is a measure of how "pointing in the same direction" the vector $Ax$ is compared to $x$. The matrix $A$ is called positive definite if $x \cdot (Ax) > 0$ for all $x \neq 0$. This gives some immediate consequences:
If this was in one dimension, what would you get?
You would get a second degree polynomial $ax^2$. Imagine now instead dimension $37$ and a function $f(x_1,x_2 ... , x_{37})=x_1^2 + x_2^2 + ... + x_{37}^2$. This is very cumbersome to type, $x^TAx$ is a generalization to higher dimension, a much more compact form to write a second degree polynomial instead of writing all terms out.