Explanation of solution for $\langle x,y\rangle = x^TAy$

Question

Show that every scalar product can be described by a symmetrical positive definite matrix. With $A\in \mathbb{R}^{N \times N}$ and $x,y \in \mathbb{R}^N$: $$\langle x,y\rangle = x^TAy.$$

---

Solution:

\begin{align} \langle x,y \rangle &= \left\langle \sum_{i}x_i e_i, \sum_{j}y_j e_j \right\rangle \\ &= \sum_{i}x_i \left\langle e_i, \sum_{j}y_j e_j \right\rangle \\ &= \sum_{i}\sum_{j} x_i y_j \langle e_i, e_j \rangle \\ &= x^TAy, \end{align} with $A_{ij} = \langle e_i, e_j \rangle$.

The defined matrix $A$ is symmetrical and positive definite because of the symmetry and positive definiteness of the scalar product.

---

So this is the solution, but I don’t understand it.

But shouldn’t it be like this?: $$\langle x,y \rangle = x_i y_i +\ldots + x_n y_n$$

So what was exactly done in the sum?

Could someone help me and explain the steps?

Answer 1

But shouldn't it be like this: $\langle x,y \rangle = x_i * y_i +....+ x_n * y_n$

This assumes the ordinary inner product, the so-called "dot product", over the standard orthonormal basis. Scalar products can take on many, many more general forms.

What the summation does is use the properties of scalar products; namely, if the underlying field is $\mathbb{R}$ (i.e. $x_i,y_i$ are all real numbers), then $\newcommand{\ip}[1]{\left\langle #1 \right \rangle} \ip{\cdot,\cdot}$ is linear in each coordinate.

Then, without any assumptions on the basis $\{e_i\}_{i=1}^N$ and just that

$$x = \sum_{i=1}^N x_i e_i \qquad y = \sum_{i=1}^N y_i e_i$$

for some $x_i,y_i \in \mathbb{R}$, we get that

$$\ip{x,y} = \sum_{i,j=1}^N x_i y_i \ip{e_i,e_j}$$

We also have not made any assumptions on how $\ip{\cdot,\cdot}$ is defined, but notice what this equation above means: however $\ip{\cdot,\cdot}$ is defined, its value on any two vectors $x,y$ is (essentially) entirely determined by how $\ip{\cdot,\cdot}$ works on the basis.

The dot product you're familiar with assumes that

$$\ip{e_i,e_j} := \delta_{i,j} := \begin{cases} 1, & i = j \\ 0, & i \ne j\end{cases}$$

but, as stated, that need not be the case.

(After all, think about what the problem is stating: many inner products can be "described" or "translated" into a positive-definitive symmetric matrix, and naturally that process should be reversible if just sometimes, i.e. in some cases you should be able to take $A$ positive-definite and symmetric, and define an inner product through its entries.)

From here, the problem defines the matrix $A := (a_{i,j})_{1 \le i,j \le N}$ by the rule

$$a_{i,j} := \ip{e_i,e_j}$$

and then just uses standard definitions of matrix operations to conclude.

Answer 2

Your $\langle x,y \rangle = x_1 \cdot y_1 +\cdots + x_n \cdot y_n$ is just one possible inner product on $\Bbb R^n$. There are, in fact, plenty of different inner products. For instance, on $\Bbb R^2$ $$ \langle x,y\rangle =x_1y_1-\frac12x_1y_2-\frac12x_2y_1+x_2y_2 $$ is also an inner product, this is the expansion that the sum does. In this case, we have $$A=\begin{bmatrix}1&-\frac12\\-\frac12&1\end{bmatrix}$$which is symmetric positive definite, and we have $$\langle e_1,e_1\rangle =\langle e_2,e_2\rangle=1\\ \langle e_1,e_2\rangle=\langle e_2,e_1\rangle=-\frac12$$

If you like, you can twist it and say there is only one inner product, but you can change the coordinate axes, without changing the inner product. In that case, if $B$ is the change-of-basis matrix, then $A=B^TB$.

The standard inner product corresponds to $A=I$.