What is the intuition behind $x^T A x$?

Question

Consider an $N$-dimensional vector $x$, and a $N \times N$ matrix $A$. Throughout linear algebra, I find the expression $x^T A x$ to be extremely common.

Is there some fundamental intuition or geometric meaning behind it?

Answer 1

Well if $A$ has entries $a_{jk}$ then $$x^TAx=\sum_{j,k}a_{jk}x_jx_k.$$

So $x^TAx$ is just a concise notation for a quadratic form.

Answer 2

The vectors "born" as a column vector, and they will be row vector after you transpose them. So the $x^TAx$ will be a scalar.

Answer 3

When matrix satisfies some nice properties, like positive definite, that expression can be viewed as a generalization of the dot product.

Answer 4

Let $\langle\cdot,\cdot\rangle$ be a bilinear form on a $n$-dimensional vector space $V$ over a field $F$, and let $\{e_1,e_2,\ldots,e_n\}$ be a basis of $V$. Let$$A=\bigl(\langle e_i,e_j\rangle\bigr)_{1\leqslant i,j\leqslant n}$$be the matrix of the bilinear form with respect to $V$. Then, if you identify each $x\in V$ with the vector of $F^n$ whose entries are the coordinates of $x$ with respect to the basis $\{e_1,e_2,\ldots,e_n\}$, then$$(\forall x\in V):\langle x,x\rangle=x^TAx.$$

Answer 5

The expression $x^TAx$ represent a quadratic form which associates at each vectors in $\mathbb{R^n}$ a scalar value by the symmetric matrix A. Indeed $y=Ax$ is a column vector and $x^Ty$ is a dot product.

Note that only symmetric matrix need to be used because any matrix M can be written as sum of a symmetric part $\frac{M+M^{T}}2$ and an antisymmetric part $\frac{M-M^{T}}2$. It is easy to verify that the quadratic form for the antisymmetric part is equal to 0.

Answer 6

I want to give some intuition: In some setups you can interpret $-x^TAx$ as energy (note the minus!).

Imagine you are given the $x$, then $x^TAx$ tells you how much $A$ is able to perturb your $x$. So, let's say that $A$ is unable to perturb $x^T$, then $x^TA\approx x^T:=y^T$. Consequently the dot product $\langle y,x\rangle$ will be almost identical to the maximal dot product $\langle x,x\rangle $ and so $-x^TAx$ will be very small. So you could think of $x$ as having little energy, as it is relatively stable against perturbations by $A$.

Conversely, if $A$ strongly perturbes $x$, then $x^TA$ will be verry different from $x^T$ and so, $-x^TAx$ will be very high meaning that the energy of this $x$ is very high.

This interpretation is actually borrowed from Hopfield Networks in which $A$ is the connectivity matrix of the network which converges for an input vector $x$ against a vector $x'$ s.t. its energy (i.e. $x'^TAx'$) is locally minimized.

Answer 7

I like to think about ${\bf x}^t {\bf A} {\bf x}$ geometrically.

Consider ${\bf x} = \{x,y,z\} \in \mathbb{R}^3$, and the special case ${\bf A}$ is the $3 \times 3$ identity matrix. Consider ${\bf x}^t {\bf A} {\bf x} = 1$ which, written out, is:

$x^2 + y^2 + z^2 = 1$, which is of course a sphere:

In short, you can think of ${\bf x}^t{\bf A}{\bf x}$ as leading to a scalar value throughout the space. When you solve for positions that give a particular value, you get a surface whose shape depends upon ${\bf A}$. Since the equation is a quadratic, you get spheres, ellipsoids and other quadratic surfaces.

If ${\bf x}^t {\bf A} {\bf x} = 4$, the sphere's radius is $2$.

If ${\bf A} = \{ \{ 3,0,0 \}, \{ 0,1,0 \}, \{0,0,1 \} \}$

then the equation is

$3 x^2 + y^2 + z^2 = 1$, or an ellipsoid:

For other positive-definite matrices the surface will be tipped and elongated appropriately due to cross terms.

If ${\bf A}$ isn't positive definite we get other quadratics, e.g., if ${\bf A} = \{ \{1,0,0 \}, \{0,-2,0 \}, \{0,0,1 \} \}$ and we solve for ${\bf x}^t {\bf A} {\bf x} = 1$:

Here are the surfaces for different values of the constant: