Orthonormal Bases

Question

I am struggling to get my head around orthonormal bases, this is the defintion in my course notes:

If anyone could clarify/explain the concept to me, it would be much appreciated. I am a university first year student, and this is for my first course in Linear Algebra.

Thanks

Answer 1

A basis is any set of vectors that are both linearly independent and span the space. If $V$ is a vector space and $b_1, ...,b_n$ are a basis of $V$ this means that

(1) any $v \in V$ can be written in the form $c_1b_1 + ...+c_nb_n$ where $c_i \in \mathbb R$ (that is saying that they span the space)

and

(2) if $c_i$ are such that $c_1b_1 + ...+c_nb_n = 0$ then $c_i$ must be $0$ for all $i$ (that's saying they are linearly independent)

Now you have a dot product in $\mathbb R^n$ and you define that you call two vectors $u=c_1b_1 + ... c_nb_n$ and $v=d_1b_1 + ...+d_nb_n$ orthogonal if and only if $u\cdot v = c_1d_1 + ... +c_nd_n = 0$. In $\mathbb R^n$ you may think of orthogonal vectors as perpendicular.

If $b_i$ are a basis then you may require them to form an orthogonal basis. For example, in $\mathbb R^3$ the vectors $(1,0,0)^T, (0,1,0)^T, (0,0,1)^T$ form an orthogonal basis as you can easily verify.

You may want to require additional nice properties. For example, you may require all the $b_i$ to have length $1$. You define that you call a set of vectors $b_i$ orthonormal if and only if they are both orthogonal and have length $1$.

Answer 2

The defining property of a basis is that every vector in the vector space can be written as a unique combination of the basis vectors. This is written as,

$$ \vec{v} = a_1 \vec{e}_1 + a_2 \vec{e}_2 + \cdots + a_n \vec{e}_n. $$

With the understanding that this can be done for every $\vec{v}$ in the vector space. The coefficients $a_k$ are the components of the vector. The vectors $\vec{e}_k$ are the basis vectors.

When you have a dot product (sometimes called an inner product) you can define the concepts of angle and magnitude in your vector spaces.

The magnitude of a vector is defined by,

$$ \vert \vec{v} \vert = \sqrt{\vec{v}\cdot \vec{v}} $$

The angle between two vectors is defined by,

$$ \cos(\theta) \equiv \frac{\vec{u}\cdot\vec{v}}{\vert \vec{u} \vert \vert \vec{v} \vert}$$

Now that we have a notion of angle we can consider the case when the vectors are orthogonal. Orthogonal means that the angle between the vectors is $90^\circ$. This condition can be written as,

$$ \vec{u} \cdot \vec{v} = 0$$

We say that a vector is normalized when its magnitude is $1$. This condition can be written as,

$$ \vec{u} \cdot \vec{u} = 1 $$

If our basis vectors are all normalized and orthogonal to each other then we say that we have a orthonormal basis. This can be written as,

$$ \vec{e}_i \cdot \vec{e}_i = 1$$

$$ \vec{e}_i \cdot \vec{e}_j = 0 \quad (i\neq j)$$

The utility of using such a basis is demonstrated by Fourier's trick which allows us to use this property to calculate the components of a vector in this basis. Consider the example,

$$ \vec{e}_2 \cdot \vec{v} = \vec{e}_2\cdot(a_1 \vec{e}_1 + a_2 \vec{e}_2 + \cdots + a_n \vec{e}_n) $$

$$ = a_1 \vec{e}_2 \cdot \vec{e}_1 + a_2 \vec{e}_2 \cdot \vec{e}_2 + \cdots + a_n \vec{e}_2 \cdot\vec{e}_n $$

$$ = a_1 (0) + a_2 (1)+ \cdots + a_n (0) $$

$$ = a_2 $$

Answer 3

This is an informal answer, but hopefully'll give you an idea of the concept. Let's pick an easy vector space, $R^2$ (which is just the familiar xy plane). Hopefully that makes sense to you that this is a vector space. If you pick any 2 points on the plane (that aren't "in line" with each other from the origin), can you see how linear combinations of those 2 points (which or course are 2 vectors in $R^2$) can "blanket" the whole xy plane? If this doesn't make sense to you, then maybe you can take a few steps back 1st before trying to bite off the concept of orthonormal bases. Otherwise, all's I'm pointing out is that these 2 vectors (points on the xy plane) can span the whole xy plane (look up "span" and digest that real good if that doesn't make sense to you).

Ok, so an orthonormal basis in this case (this case of the xy plane), could just be the points $v_1=(0,1)$ and $v_2=(1,0)$. If you try these 2 points, on the definition, see if they hold? Try that 1st with this example, and see if that sheds some light. I'm sure other questions will pop up in your head, but then at least you'll have advanced the ball. Hope that helps a bit.