Orthogonality of a matrix where inner product is not the dot product

Question

Here are the definitions I am using:

1. Orthogonality: Two vectors $x$ and $y$ are orthogonal iff $\langle x,y \rangle=0$.
2. Orthonormal: If two vectors $x$ and $y$ are orthogonal and $||x|| = 1 = ||y||$ then $x$ and $y$ are orthonormal.
3. Orthogonal Matrix: A square matrix $\mathbf A \in \Bbb {R}^{n \times n}$ is an orthogonal matrix iff its columns are orthonormal so that $\mathbf {AA^T = I = A^TA}$.

Suppose we use an inner product defined by:

$$\langle x, y \rangle = x^T \begin{bmatrix} 2 & 1 & 0 \\ 1 & 2 & -1 \\ 0 & -1 & 2 \\ \end{bmatrix}y$$

We define $e_1, e_2, e_3$ as the standard basis vectors in $\Bbb {R}^{3}$.

When we evaluate the inner products we find:

$\langle e_1, e_3 \rangle = 0$

$\langle e_1, e_2 \rangle = 1$

$\langle e_3, e_2 \rangle = -1$

So clearly using this inner product, only the basis vectors $e_1$ and $e_3$ are orthogonal.

Separately, if I construct a matrix made by the basis vectors:

$\mathbf A = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}$

It holds true that $\mathbf {AA^T = I = A^TA}$.

Is the matrix $\mathbf A$ orthogonal? Because the columns (using our definition of the inner product) are not actually orthogonal and therefore cannot be orthonormal, but the multiplication to identity holds.

Answer 1

If you need to check whether the columns are orthogonal or not for an inner product defined by an inner product, it is not $A A^T$ that you must consider.

Indeed with: $$\langle x, y \rangle =x^T M y$$

you have $A=(e_1, \ldots, e_n)$ is orthonormal iff: $$ \forall i,j \, \, \langle e_i , e_j \rangle=e_i^T M e_j =\delta_{i,j}$$ i.e in matrix notation:

$$ A^T \color{red} M A = I$$

for the standard inner product $M=I$ so you obtain $A^T A$, but in your example: $$M=\begin{bmatrix} 2 & 1 & 0 \\ 1 & 2 & -1 \\ 0 & -1 & 2 \\ \end{bmatrix}$$

so $A^T M A \neq I$.

Answer 2

Yes, the matrix is orthogonal. All orthogonal matrices have columns with orthonormal vectors with respect to the dot product, regardless of your choice of inner product.

There is something called an orthogonal linear transformation, that is, $TT^* = T^*T = I$. The matrix representation of these linear transformations with respect to an orthonormal basis (any inner product) is always an orthogonal matrix.

In your example, you just showed that the identity transformation is orthogonal in all choices of inner product.