Some aspects of inner products in $\mathbb R^3$

Question

I read several descriptions about inner products recently due to an exercise (here it is no longer active so I like to ask again). I am still very confused about this concept. Any clarification will be greatly appreciated.

1. The very first definition of inner product I encountered is $\langle v, w\rangle =v_1w_1+v_2w_2+v_3w_3$. Or in general, it is a function defined on $\mathbb R^3\times\mathbb R^3$ into $\mathbb R$ which satisfies certain positivity and linearity conditions. I have no problem in understanding this definition.
2. Then people talk about the set of inner products in $\mathbb R^3$. It seems that an inner product can be represented by a matrix. Why? And how? This does not make sense to me because if we write inner product in vector notation, then we have $\langle v, w\rangle=v^Tw$. There is no matrix in this notation, is there? Moreover, what does it mean by a set of inner products? Does that mean a set of functions (since inner product can be treated as a function as indicated in 1)?
3. What is the natural topology on the set of inner products? Can I say that the topology is the sub product topology of $\mathbb R^3\times \mathbb R^3$?

Answer 1

With regards to your second point, one can actually view the usual inner product $\langle v, w \rangle = v^T w$ as a special case of the more general $\langle v, w \rangle = v^T A w$ where $A$ is a matrix satisfying sufficient conditions to make this an inner product. The usual inner product is just this with $A = I$.

Answer 2

Think of $\langle v, w \rangle = v^\top w = v^\top I w$, where $I$ is the $3 \times 3$ identity matrix. Then, a weighted inner product on $\mathbb{R}^3$ can have the form $v^\top M w$, where $M$ is a matrix. If $M$ is a $3 \times 3$ diagonal matrix, then the arising inner product is the weighted euclidean inner product on $\mathbb{R}^3$.

The set of inner products on $\mathbb{R}^3$ is a subset of the set of functions $f : \mathbb{R}^3 \times \mathbb{R}^3 \to \mathbb{R}$. If every inner product on $\mathbb{R}^3$ is of the form $v^\top M w$ for some matrix $M \in \mathbb{M}_{3,3}(\mathbb{R})$, then you can regard the set of matrices giving rise to these products as the set of inner products themselves.

I am unaware of any useful topology on this space of functions. Sorry and hope this helped!

Answer 3

The space of general inner products on $\mathbb{R}^n$

On $\mathbb{R}^n$, the most general definition of an inner product is that an inner product is some bilinear map $F \,:\, \mathbb{R}^n\times \mathbb{R}^n \to \mathbb{R}$. It's easy to see that the bilinearity implies that $F$ is fully determined by the images of pairs of basis vectors of $\mathbb{R}^n$, i.e. if $(b_i)_{1\leq i \leq n}$ is a basis of $\mathbb{R}^n$, then $F$ is fully determined by the scalars $$ f_{i,j} = F(b_i,b_j) \text{, $1 \leq i,j \leq n$.} $$

It's equally straight-forward to see that every such choice of scalars defines an inner product, because we can always set $$ F(x,y) = \sum_{i=1}^n \sum_{j=1}^n f_{i,j} x_i y_j \quad\text{where } x = \sum_{i=1}^n x_i b_i,\, y = \sum_{j=1}^n y_j b_j \text{.} $$ (i.e. $(x_i)$ respectively $(y_i)$ is the coordinatization of $x$ respectively $y$ in the basis $(b_i)$) This also shows that inner products correspond to $n\times n$ matrices, because if we interpret the $f_{i,j}$ as the coefficients in such a matrix, we get $$ F(x,y) = x^T Ay \text{.} $$

Thus, the inner products on $\mathbb{R}^n$ form a vector space isomorphic to $\mathbb{R}^{n^2}$. If you place additional restrictions on inner products, such as symmetry (i.e., $F(x,y) = F(y,x)$), positive definiteness (i.e. $F(x,x) > 0$ if $x \neq 0$), the resulting space is some subset of $\mathbb{R}^{n^2}$. For some restrictions (like symmetry), you get a subspace, while others (like positive definitness) yield more complex forms of subsets. Which is why your teacher said that you get some manifold.

There are multiple ways to define a topology on the vector space of inner products. Since it's isomorphic to $\mathbb{R}^{n^2}$, one way is to simply use the usual euclidean norm. Another is the define a kind of operator norm, i.e. set $$ \|F\|_{\textrm{op}} = \limsup_{x,y \in \mathbb{R}^n} \frac{|F(x,y)|}{\|x\|\cdot \|y\|} \text{.} $$ By using the representation of $F$ as $f_{i,j}$ for some basis $(b_i)$ of $\mathbb{R}^n$ from above, you can show that an equivalent (though not necessarily identical, I think) norm can be defined as $$ \|F\|_\infty = \max_{1 \leq i,j \leq n} \left|f_{i,j}\right| $$

The subspace $S_n$ of symmetric inner products on $\mathbb{R}^n$

If we fix a basis $(b_i)$ of $\mathbb{R}^n$, symmetry of an inner product $F$ means for the corresponding matrix $A$ that $x^T Ay = y^T A x$. By using that $(uv)^T = v^Tu^T$ for arbitrary $k\times l$ matrices $u$ and arbitrary $l\times m$ matrices $v$, we get $$ y^T A y \overset!= x^T A y = \left(x^T A y\right)^T = (Ay)^T x = y^T A^T y \Rightarrow A = A^T $$ i.e. that $A$ must be a symmetric matrix (how nice that the two meanings of symmetric for matrices and inner products coincide! No risk of confusion there!). It's easy to see that the set $S_n$ of symmetric $n \times n$ matrices form a subspace of $\mathbb{R}^{n^2}$ with $$ \dim S_n = n + \frac{n^2 - n}{2} = \frac{n(n+1)}{2} \text{.} $$ Just observe that a symmetric matrix is fully determined by the coefficient $f_{i,j}$ with $i \leq j$, and that any such choice of coefficients can be extended to a symmetric matrix by setting $f_{i,j} = f_{j,i}$ for $i > j$.

The set $S_n^+$ of symmetric positive definite inner products on $\mathbb{R}^n$

Now we further restrict $S_n$ to the set of positive definite and symmetric matrices $S_n^+$, i.e. to $$ S_n^+ = \{A \in S_n \,:\, x^TA x > 0 \text{ for all $0 \neq x \in \mathbb{R}^n$}\} \text{.} $$ If $A,B \in S_n^+$, then for $C = A+B$ and $x \neq 0$ we have $$ x^T C x = x^T (A+B)x = x^T(Ax + Bx) = \underbrace{x^TAx}_{> 0} + \underbrace{x^TBx}_{> 0} > 0 $$ so $S_n^+$ is closed under sums. Similarly, if $A \in S_n^+$, $x \neq 0$ and $0 < \lambda \in \mathbb{R}$, then for $C = \lambda A$ we have $$ x^T C x = x^T (\lambda A) x = \underbrace{\lambda}_{> 0}(\underbrace{x^TAx}_{> 0}) > 0 \text{.} $$ But if $\lambda = 0$, then $\lambda A = 0 \neq S_n^+$, and for $0 > \lambda \in \mathbb{R} $ we have $x^T\lambda A x < 0$ for $x \neq 0$. $S_n^+$ is therefore not closed under scalar multiplication. So, unfortunately, $S_n^+$ is not a subspace of $S_n$.

The set $S_n^+$ as a manifold

So what, then, can we say about $S_n^+$?. Let's again pick some $A \in S_n^+$, and let $$ \mu_A := \inf_{\|x\| = 1} x^T A x \text{.} $$ Since the unit sphere $\{x \in \mathbb{R}^n \,:\, \|x\|=1\}$ is a compact subset of $\mathbb{R}^n$, and since the map $x \mapsto x^T Ax$ is continuous, $\mu_A > 0$. (Because if not, pick a convergent sequence $x_n \to x$ with $\|x_n\|=1$ and $x_n^TAx_n < \frac{1}{n}$, and use the continuity to show that $x^TAx = 0$. The compactness of the unit sphere plays a crucial role here!).

We now take some arbitrary symmetric matrix $B \in S_n$, and look at $C = A + \lambda B$. As a sum of symmetric matrices, $C$ is obviously symmetric. Let $0 \neq x \in \mathbb{R}^n$ be an arbitrary vector, let $l = \|x\| > 0$, and let $x_1 = \frac{x}{l}$. In other words, $x_1$ is the unique vector on the unit sphere with $x = l x_1 $. Then $$ x^T C x = l^2\left(x_1^T(A + \lambda B)x_1\right) = l^2\big(\underbrace{x_1^TAx_1}_{\geq \mu_A} + \lambda \underbrace{x^TBx}_{\in[-M_B,M_B]}\big) \geq l^2\left(\mu_A - \lambda M_B\right) > 0 \text{ if $\lambda < \frac{\mu_A}{M_B}$} $$ where $$ M_B = \sup_{\|x\|=1} |x^TBx|. \quad \text{($M_B < \infty$, again because $x \mapsto x^TBx$ is continuous)} $$

This is very interesting! It means that we we start from some element in $S_n^+$, we can add scaled versions of arbitrary symmetric matrices $B$ and the result is still positive definite, so long as we keep the scaling factor small enough. We can strenghten this result a bit more. Since $S_n$ is finite-dimensional, we can pick some basis $(B_i)$ of $S_n$, and set $$ M = \max_{1 \leq i \leq \dim S_n} M_{B_i} \quad\text{i.e.}\quad M \leq M_{B_i} \text{ for all basis matrices $B_i$.} $$

It the follows that if $A \in S_n^+$, all the matrices $$ A + \sum_{i=1}^{\dim S_n} \mu_i B_i \quad\text{ where } \sum_{i=1}^{\dim S_n} |\mu_i| < \frac{\mu_A}{M} $$ are positive definite!. This shows that locally, $S_n^+$ has the same dimension as $S_n$, because if we restrict ourselves to some small neighbourhood of a particular symmetric and positive definite $A$, that neighbourhood "looks" just like $S_n$.

The case $\mathbb{R}^2$

In the case of $\mathbb{R}^2$, $\dim S_n = 3$. For $S \in S_2^+$, we must have for all $x_1,x_2$ where one value is non-zero that $$ (x_1,x_2) A (x_1,x_2)^T = a_{11} x_1^2 + a_{22} x_2^2 + 2a_{12} x_1x_2 > 0 \text{.} $$ The cases $(x_1,x_2) = (1,0)$ respectively $(x_1,x_2) = (0,1)$ show that this requires $a_{11} > 0$, $a_{22} > 0$. Rewriting as $$ a_{11} x_1^2 + a_{22} x_2^2 + 2a_{12} x_1x_2 = \left(\sqrt{a_{11}}x_1 + \sqrt{a_{22}}x_2\right)^2 - 2\sqrt{a_{11}a_{22}}x_1x_2 + 2a_{12}x_1x_2 $$ yields for $a_{12}$ the condition $$ |a_{12}| < \sqrt{a_{11}a_{12}} \text{.} $$

We change the base of $S_3$ such that the new coordinate $x,y,z$ obey $x + y = a_{11}$, $x - y = a_{22}$ and $z = a_{12}$, then transformed conditions on $x,y,z$ are $$\begin{eqnarray} x &>& 0 \\ |y| &<& x \\ |z| &<& \sqrt{x^2 - y^2} \text{.} \end{eqnarray}$$ The set $S_3^+$ is thus a cone which is rotationally symmetric around the ray $x > 0, y=z=0$, which in the original coordinates $a_{11},a_{12},a_{12}$ means the ray $a_{11} = a_{22} = c$.

Answer 4

Abstractly, inner products are special cases of something from geometry called a symmetric bilinear form on a vector space. A bilinear form on and $F$ vector space $V$ is just a function $B:V\times V\to F$ satisfying

• $B(x,y)=B(y,x)$
• $B(x+y,z)=B(x,z)+B(y,z)$
• $B(\lambda x,y)=B( x,\lambda y)=\lambda B(x,y)$

You'll notice the positive definiteness criterion ($B(x,x)>0$ for all $x\neq 0$) is missing.

Another thing that's missing is any mention of coordinates or components of vectors! If you pick any basis at all, then you can bulid a matrix $X$, necessarily a symmetric matrix, such that $\vec{x}^\top X \vec{y}=B(x,y)$, where $\vec{x}$ is the coordinate representation of $x$ in that particular basis.

When you are working in the simple basis with $(1,0,0)$, $(0,1,0)$ and $(0,0,1)$ and the usual inner product, then $X$ is just the identity matrix. But keep in mind $X$ doesn't have to be the identity matrix.

Now a natural question is this:

If I'm willing to undergo a change of basis, is there any way I can simplify $X$?

For the case of $F=\Bbb R$, you can always change basis so that the matrix for the bilinear form is zero everywhere except on the diagonal, where it can have $0$'s, $1$'s and/or $-1$'s. If you additionally require that $B(x,x)>0$ for all $x\neq 0$, then $X$ can be changed into the identity matrix. You can learn a little about this from the wiki article on metric signatures.

Also keep in mind that here are some fields $F$ that are not as nice as $\Bbb R$, and won't admit such nice simplifications.

---

So how can you view the set of inner products (for, say, $\Bbb R$)? For a fixed basis, you can view the set of possible symmetric bilinear forms on $\Bbb R^n$ as the set of $n\times n$ symmetric matrices. If you want to focus on positive definite forms only, then you have to further restrict this to the set of positive definite symmetric matrices.

Any positive definite symmetric matrix at all will define an inner product on the vectors in the basis you chose, and you can see by playing around with some examples that they give different forms. For example, you could take the matrix $X=\begin{bmatrix}1&0\\0&2\end{bmatrix}$ and examine $\vec{v}^\top=(0,1)$. Notice that $\vec{v}^\top X\vec{v}=2 $, but $\vec{v}^\top I_2 \vec{v}=1$, so the identity matrix $I_2$ and $X$ are definitely giving two different inner products on $\Bbb R^2$ for the same input vector.

---

The set of symmetric matrices $S$, and the set of positive definite symmetric matrices $S^+$ could be given the subspace topology from whatever topology you choose on the set of $n\times n$ matrices. Perhaps one final thing to note is that while $S$ is a linear subspace of the $n\times n$ matrices, $S^+$ isn't.