Inner product and orthogonality in non orthogonal basis

Question

According to the definition of orthogonality (on finite vector spaces),

Given an inner product space, two vectors are orthogonal if their inner product is zero.

So as an example, assuming the inner product is the "the standard" Euclidean inner product, two vectors (1,0) and (0,1), in $\mathbb{R}^2$, are obviously orthogonal in standard basis. However, if we change the basis to another basis of two linearly independent vectors which are not orthogonal in standard $\mathbb{R}^2$ basis, then these new basis vectors would be also represented as, (1,0) and (0,1) in the new basis. So keeping the same Euclidean inner product, they would be orthogonal in the new basis, although, they are not orthogonal in standard $\mathbb{R}^2$ basis. This would mean that orthogonality depends on the basis chosen, which seems "strange" to me?

Can you please explain what is wrong with in the above reasoning (assuming it is wrong).

Thanks.

Note, I have read the thread with the very similar title, change of basis and and inner product in non orthogonal basis but that doesn't answer my question

Answer 1

The key point to understand here is that you really are dealing with two $\mathbb R^2$ here, although it's not that obvious when using the standard basis.

The first $\mathbb R^2$ is your vector space. Let's write this vector space and everything in it in blue. This first $\color{blue}{\mathbb R^2}$ is equipped with a vector space structure and additionally with the dot product $\color{blue}{\mathbf x\cdot\mathbf y = x_1y_1+x_2y_2}$.

Now, as soon as you choose a basis $\{\color{blue}{\mathbf b_1},\color{blue}{\mathbf b_2}\}$ that spans ${\mathbb R^2}$, you can write every vector $\color{blue}{\mathbf x}\in\color{blue}{\mathbb R^2}$ in an unique way as $\color{blue}{\mathbf x}=\color{red}{\xi_1}\color{blue}{\mathbf b_1}+\color{red}{\xi_2}\color{blue}{\mathbf b_2}$. Note that $\color{red}{\xi_1}$ and $\color{red}{\xi_2}$ are not the components of the vector in $\color{blue}{\mathbb R^2}$, but are base dependent.

But, you need always two of them, and when doing vector addition and multiplication with scalar, you'll find they behave exactly like the components of a vector should behave. Therefore, it does make sense to consider them as part of a $\color{red}{\mathbb R^2}$, which, however, is a different $\mathbb R^2$ than the original $\color{blue}{\mathbb R^2}$ we started with. In particular, the coordinate $\color{red}{\mathbb R^2}$ is not pre-equipped with an inner product.

The basis then defines a linear map $\beta$ from the coordinate $\color{red}{\mathbb R^2}$ to the original $\color{blue}{\mathbb R^2}$ given by $$\beta(\color{red}{\boldsymbol\xi})=\color{red}{\xi_1}\color{blue}{\mathbf b_1} + \color{red}{\xi_2}\color{blue}{\mathbf b_2}.$$

Now, remember that I said that the coordinate $\color{red}{\mathbb R^2}$ is not pre-equipped with an inner product. That doesn't mean we cannot give it one. That said, if we do give it an inner product, then we want to do it in a way that the product is preserved by the map $\beta$, that is, you want to have $$\color{red}{\langle \boldsymbol\xi,\boldsymbol\eta\rangle} = \beta(\color{red}{\boldsymbol\xi})\color{blue}{\cdot}\beta(\color{red}{\boldsymbol\eta})$$ where $\color{red}{\langle \boldsymbol\xi,\boldsymbol\eta\rangle}$ denotes the inner product in the coordinate $\color{red}{\mathbb R^2}$. By inserting the explicit formula of $\beta$, one easily sees that $$\color{red}{\langle \boldsymbol\xi,\boldsymbol\eta\rangle} = \sum_{j,k=1}^2 (\color{blue}{\mathbf b_j\cdot\mathbf b_k})\color{red}{\xi_j\eta_k}.$$

Now, note that if $\{\color{blue}{\mathbf b_1},\color{blue}{\mathbf b_2}\}$ is not an orthogonal basis, then $\color{red}{\langle \boldsymbol\xi,\boldsymbol\eta\rangle}\ne\color{red}{\xi_1\eta_1}+\color{red}{\xi_2\eta_2}$; and, we confirm that the inner product on the coordinate $\color{red}{\mathbb R^2}$ explicitly depends on the chosen basis $\{\color{blue}{\mathbf b_1},\color{blue}{\mathbf b_2}\}$. This is not surprising, because the vector in $\color{blue}{\mathbb R^2}$ that those coordinates (i.e., $\color{red}{\xi_1,\xi_2,\eta_1,\eta_2}$) describe depends on the basis chosen. Thus, in general, different pairs of vectors have different inner products.

Note that by definition of the inner product, with $\beta(\color{red}{\boldsymbol\xi})=\color{blue}{\mathbf x}$ and $\beta(\color{red}{\boldsymbol\eta})=\color{blue}{\mathbf y}$ it is of course still true that $$\color{red}{\langle \boldsymbol\xi,\boldsymbol\eta\rangle} = \color{blue}{\textbf x\cdot\textbf y} = \color{blue}{x_1y_1} + \color{blue}{x_2y_2}.$$ But in general, $\color{blue}{x_1y_1} + \color{blue}{x_2y_2} \ne \color{red}{\xi_1\eta_1}+\color{red}{\xi_2\eta_2}$.

However, if you chose the standard basis $\color{blue}{\mathbf b_k}{\mathbf e_k}$ then you obviously have $\color{red}{\xi_k}=\color{blue}{x_k}$ and $\color{red}{\langle \boldsymbol\xi,\boldsymbol\eta\rangle} = \color{blue}{x_1y_1+x_2y_2} = \color{red}{\xi_1\eta_1}+\color{red}{\xi_2\eta_2}.$ This is precisely how come it is easy to miss the fact that you are working with two different $\mathbb R^2$ when using the standard basis.

Answer 2

$\left( \begin{array}{c} 1 \\ 0 \\ \end{array} \right)$ and $\left( \begin{array}{c} 0 \\ 1 \\ \end{array} \right)$ are the coordinates of the two basis vectors, not the actual vectors .

So $\left( \begin{array}{c} 1 \\ 0 \\ \end{array} \right)$ represents $\vec{e_1}$ and $\left( \begin{array}{c} 0 \\ 1 \\ \end{array} \right)$ represents $\vec{e_2}$ .

In an orthonormal basis we have the dot product of basis vectors is zero : $\vec{e_1} \cdot \vec{e_2} =0$.

But in some other basis it might be that : $\vec{e'_1} \cdot \vec{e'_2} \neq 0$ even though the coordinates of the basis vectors are the same in both cases.

In other words : in the $\left\{ \vec{e'_1}, \vec{e'_2}\right\}$ basis , $\left( \begin{array}{c} 0 \\ 1 \\ \end{array} \right) \cdot \left( \begin{array}{c} 1 \\ 0 \\ \end{array} \right) \neq 0 $ . In terms of coordinates you could say that this is because the matrix for the dot product is not : $\left( \begin{array}{c , c} 1 & 0 \\ 0 & 1 \\ \end{array} \right)$ in that basis.

Update :

The question is about an inner product space and how it is affected by a change of basis.

As an example we first look at the normal 2-D Euclidean space with orthonormal basis. The basis vectors now are represented in coordinates by these two matrices:

$\left( \begin{array}{c} 0 \\ 1 \\ \end{array} \right)$ and $\left( \begin{array}{c} 1 \\ 0 \\ \end{array} \right)$

The inner product is represented by this matrix : $\left( \begin{array}{c , c} 1 & 0 \\ 0 & 1 \\ \end{array} \right)$ so that the inner product of $\left( \begin{array}{c} a \\ b \\ \end{array} \right)$ and $\left( \begin{array}{c} c \\ d \\ \end{array} \right)$ becomes : $\left( \begin{array}{c , c} 1 & 0 \\ 0 & 1 \\ \end{array} \right)\left( \begin{array}{c} a \\ b \\ \end{array} \right)\left( \begin{array}{c} c \\ d \\ \end{array} \right) = \left( \begin{array}{c} a & b\\ \end{array} \right)\left( \begin{array}{c} c \\ d \\ \end{array} \right)=ac+bd$

Now we express this same system with respect to a new basis : Let's say our new basis vectors become (expressed in terms of the old orthonormal basis vectors):

$\left( \begin{array}{c} 0 \\ 1 \\ \end{array} \right)$ and $\left( \begin{array}{c} 1 \\ 1 \\ \end{array} \right)$ . These are their coordinates with respect to the 'old' $\left\{ \vec{e_1}, \vec{e_2}\right\}$ basis .

But the SAME two new basis vectors expressed in terms of the new basis are of course : $\left( \begin{array}{c} 0 \\ 1 \\ \end{array} \right)$ and $\left( \begin{array}{c} 1 \\ 0 \\ \end{array} \right)$ , so these are their coordinates with respect to the new $\left\{ \vec{e'_1}, \vec{e'_2}\right\}$ basis . The old $\vec{e_1} , \vec{e_2} $ vectors with respect to the new coordinate system ($\left\{ \vec{e'_1}, \vec{e'_2}\right\}$) become: $\left( \begin{array}{c} 1 \\ 0 \\ \end{array} \right)$ and $\left( \begin{array}{c} -1 \\ 1 \\ \end{array} \right)$

Changing the basis also means that coordinates of all other vectors and matrices in the old system need to be transformed to the new $\left\{ \vec{e'_1}, \vec{e'_2}\right\}$ system :

A : Old system $\left\{ \vec{e_1}, \vec{e_2}\right\}$ :

$\vec{e_1}=\vec{e'_1}$
$\vec{e_2}=\vec{e'_2}-\vec{e'_1}$

$\vec{a}= \left( \begin{array}{c} a_1 \\ a_2 \\ \end{array} \right)$ , $\vec{b}= \left( \begin{array}{c} b_1 \\ b_2 \\ \end{array} \right)$

The dot product : $\vec{a}\cdot\vec{b}=a_1b_1\vec{e_1}\cdot\vec{e_1}+a_2b_2\vec{e_2}\cdot\vec{e_2}$ $\enspace$ , $\enspace$ matrix form dot product : $\left( \begin{array}{c , c} 1 & 0 \\ 0 & 1 \\ \end{array} \right)$

B : New system $\left\{ \vec{e'_1}, \vec{e'_2}\right\}$ :

$\vec{e'_1}=\vec{e_1}$
$\vec{e'_2}=\vec{e_1}+\vec{e_2}$

$\vec{a}= \left( \begin{array}{c} a_1' \\ a_2' \\ \end{array} \right) = a_1\vec{e'_1} + a_2(\vec{e'_2}-\vec{e'_1}) \implies a_1'=a_1-a_2 \enspace , \enspace a_2' = a_2 $
$\vec{b}= \left( \begin{array}{c} b_1' \\ b_2' \\ \end{array} \right) = b_1\vec{e'_1} + b_2(\vec{e'_2}-\vec{e'_1}) \implies b_1'=b_1-b_2 \enspace ,\enspace b_2' = b_2 $

The dot product : $\vec{a}\cdot\vec{b}=a_1b_1+a_2b_2=(a_1'+a_2')(b_1'+b_2')+a_2'b_2' =a_1'b_1'+a_1'b_2'+ a_2'b_1'+ 2a_2'b_2'= \left( \begin{array}{c} a_1'+a_2' & a_1'+2a_2' \\ \end{array} \right)\left( \begin{array}{c} b_1' \\ b_2' \\ \end{array} \right) \implies $

Transformed matrix for the dot product becomes : $\left( \begin{array}{c , c} 1 & 1 \\ 1 & 2 \\ \end{array} \right)$

So in the new coordinates $\vec{e_1}\cdot\vec{e_2}$ is still zero : $\left( \begin{array}{c , c} 1 & 1 \\ 1 & 2 \\ \end{array} \right)\left( \begin{array}{c} 1 \\ 0 \\ \end{array} \right)\left( \begin{array}{c} -1 \\ 1 \\ \end{array} \right)=\left( \begin{array}{c} 1 & 1 \\ \end{array} \right)\left( \begin{array}{c} -1 \\ 1 \\ \end{array} \right) =0$

And $\vec{e_1'}\cdot\vec{e_2'}$ is : $\left( \begin{array}{c , c} 1 & 1 \\ 1 & 2 \\ \end{array} \right)\left( \begin{array}{c} 1 \\ 0 \\ \end{array} \right)\left( \begin{array}{c} 0 \\ 1 \\ \end{array} \right)=\left( \begin{array}{c} 1 & 1 \\ \end{array} \right)\left( \begin{array}{c} 0 \\ 1 \\ \end{array} \right) =1$

Compare this last result to $\vec{e_1'}\cdot\vec{e_2'}$ in the old system: $\left( \begin{array}{c , c} 1 & 0 \\ 0 & 1 \\ \end{array} \right)\left( \begin{array}{c} 1 \\ 0 \\ \end{array} \right)\left( \begin{array}{c} 1 \\ 1 \\ \end{array} \right)=\left( \begin{array}{c} 1 & 0 \\ \end{array} \right)\left( \begin{array}{c} 1 \\ 1 \\ \end{array} \right) =1$

We conclude that when we transform the coordinates of the dot product matrix to the new basis $\left\{ \vec{e'_1}, \vec{e'_2}\right\} = \left\{ \left( \begin{array}{c} 1 \\ 0 \\ \end{array} \right), \left( \begin{array}{c} 1 \\ 1 \\ \end{array} \right)\right\}$ we get the new dot product coordinates : $\left( \begin{array}{c , c} 1 & 1 \\ 1 & 2 \\ \end{array} \right)$ and the dot products of all vectors remain unchanged as they should .

Another way to calculate the coordinates for the transformed dot product matrix is realizing that the dot product consists of combinations of two 'row'-basisvectors.

Row vectors and column vectors :

$\vec{e_{i}}$ is column vector , $\vec{e^{i}}$ is row vector.

Column vectors and row vectors together form numbers : $\vec{e^{i}}(\vec{e_{j}}) = \vec{e_{i}}(\vec{e^{j}}) = \vec{e'_{i}}(\vec{e'^{j}})=\vec{e'^{i}}(\vec{e'_{j}})=\delta_{ij}$ where $\delta_{ij}$ is the Kronecker delta. This combination of column and row vectors to form a number is NOT the dot product and it IS coordinate independent.

Because column and row vectors together must produce coordinate independent numbers we can derive their transformation properties :

$\vec{e^1}=a\vec{e'^1} +b\vec{e'^2}$
$\vec{e^2}=c\vec{e'^1} +d\vec{e'^2}$

$1=\vec{e^2}(\vec{e_2})=\vec{e^2}(\vec{e'_2}-\vec{e'_1}) = (c\vec{e'^1} +d\vec{e'^2})(\vec{e'_2}-\vec{e'_1}) =-c+d $

$1=\vec{e^1}(\vec{e_1})= \vec{e^1}(\vec{e'_1}) = (a\vec{e'^1} +b\vec{e'^2})(\vec{e'_1}) =a $

$0=\vec{e^2}(\vec{e_1})= \vec{e^2}(\vec{e'_1}) = (c\vec{e'^1} +d\vec{e'^2})(\vec{e'_1})=c $

$0=\vec{e^1}(\vec{e_2})= \vec{e^1}(\vec{e'_2}-\vec{e'_1}) = (a\vec{e'^1} +b\vec{e'^2})(\vec{e'_2}-\vec{e'_1}) =-a+b$

It follows that :

$\vec{e^1}= \vec{e'^1} + \vec{e'^2}$
$\vec{e^2}= \vec{e'^2}$

The dot product ($DP$) expressed in the old row-basisvectors is :

$DP=\vec{e^{1}}\otimes \vec{e^{1}}+\vec{e^{2}}\otimes \vec{e^{2}}$

In terms of the new basis vectors this becomes :

$DP=(\vec{e'^1} + \vec{e'^2})\otimes (\vec{e'^1} + \vec{e'^2})+\vec{e^{2}}\otimes \vec{e^{2}} \implies$

$DP= \vec{e'^1}\otimes \vec{e'^1} + \vec{e'^1}\otimes \vec{e'^2} +\vec{e'^2}\otimes \vec{e'^1} + 2\vec{e'^2}\otimes \vec{e'^2} = \left( \begin{array}{c , c} 1 & 1 \\ 1 & 2 \\ \end{array} \right) $

Answer 3

You seem to be confusing the notion of an inner product and a basis.

A (real) inner product on a vector space $V$ is a bilinear function $\langle \cdot | \cdot \rangle\colon V\times V\to {\bf R}$ satisfying the standard axioms of positive definiteness and symmetry. It is this inner product that we use to determine whether a given tuple of vectors is orthogonal or orthonormal, not any basis.

Now, if $V$ is finite dimensional and $(v_1,\ldots, v_n)$ is a basis of $V$, there is a unique inner product such that $(v_1,\ldots, v_n)$ is an orthonormal basis. But the converse is far from truth: given any inner product on a space of dimension at least two, there are infinitely many orthonormal bases.

Now, if you have a space $V$ with a fixed inner product $\langle \cdot | \cdot \rangle_1$, you can take a basis $(v_1,\ldots, v_n)$ of $V$ and find the unique inner product $\langle \cdot | \cdot \rangle_2$ such that $(v_1,\ldots,v_n)$ are orthonormal with respect to it, but unless $(v_1,\ldots,v_n)$ are orthonormal with respect to $\langle \cdot | \cdot \rangle_1$, the two inner products are distinct, so it should come as no surprise that the notions of "orthonormal" differ (precisely because of what I have mentioned in the last paragraph about the inner product being unique given a basis which we want to be orthonormal).

Interestingly, it is actually true that any two inner products have a common orthogonal basis, but nonetheless, the notions of orthogonality do not coincide for distinct inner products, either. In other words, while there are special bases which are orthogonal jointly for both inner products, there are always bases which are orthogonal in one, but not in the other basis (for distinct inner products).