Does the phrase "orthogonal" mean the same thing when used in the terms "orthogonal function" and "orthogonal vector"?

Question

I was reading about Fourier series when I came across the term "orthogonal" in relation to functions.

http://tutorial.math.lamar.edu/Classes/DE/PeriodicOrthogonal.aspx#BVPFourier_Orthog_Ex2

I've never heard of this. The idea that two vectors are orthogonal makes sense to me because I can imagine, for instance, $\vec{a}=(1,0)$ and $\vec{b}=(0,1)$, such that $\vec{a} \cdot \vec{b} = (1)(0) + (0)(1) = 0$.

But no simple picture comes to mind for functions. Wikipedia wasn't very helpful for me.

http://en.m.wikipedia.org/wiki/Orthogonal_functions

Can someone explain what this concept is and give a simple example?

Remark:

My intuition says maybe intersecting lines would be an example of two orthogonal functions.

$f(x) = x$ $g(x) = -x$

But that's just a shot in the dark and I don't think that makes sense because the integral is just $\int -x^2 = - \frac{x^{3}}{3} + C$, which isn't zero.

Answer 1

Consider two vectors in $3$-dimensional space: \begin{align} & (2,3,7) \\[6pt] & (1,-3,1) \end{align} If you draw an accurate picture of two arrows pointing outward from the origin, they look as if they are at right angles to each other, as indeed they are. But what if we view the same array of numbers as three points in the plane: $$ \left(\begin{array}{c} 2 \\ 1 \end{array}\right), \left(\begin{array}{c} 3 \\ -3 \end{array}\right), \left(\begin{array}{c} 7 \\ 1 \end{array}\right) $$ Then they look like this: $$ \begin{array}{cccccc} \bullet & & & & & \bullet \\ \phantom{\bullet} \\ \phantom{\bullet} \\ \phantom{\bullet} \\ & & \bullet \end{array} $$ Where is the orthogonality now? When you look at, for example, the sine and cosine functions, you are looking at something like this second graph, where the geometric relation of orthogonality is not visible. It becomes visible only when the shapes of the graphs are not visible: just plot the point $(a,b)$ for $a\cos x+b\sin x$ and $(c,d)$ for $c\cos x+d\sin x$, and see whether the vectors $(a,b)$ and $(c,d)$ are orthogonal to each other.

But next we have the problem of a space of many dimensions, which our ordinary sense don't help us see. But lots of things about the geometry are the same, and that geometry can be applied to such spaces.

Answer 2

Consider a vector space of functions. To fix our ideas, let's say it's the space of all continuous real-valued functions on domain $-\pi\le x\le\pi$. We define the "dot product" $$f\cdot g=\int_{-\pi}^\pi f(x)g(x)\,dx\ .$$ It seems reasonable to call this a "dot product" because it shares the fundamental properties of the usual dot product of vectors in $\Bbb R^n$:

• $f\cdot g=g\cdot f$;
• $(f_1+f_2)\cdot g=(f_1\cdot g)+(f_2\cdot g)$;
• $(\lambda f)\cdot g=\lambda(f\cdot g)$;
• $f\cdot f\ge0$;
• if $f\cdot f=0$ then $f=0$: note that "$f=0$" means "$f(x)=0$ for all $x$".

I am sure you can very easily prove the first four of these, the fifth is a little harder. Since we are treating this definition as being similar to a dot product, it makes sense to define two functions to be orthogonal if the dot product is zero. For eaxmple, $$(\cos x)\cdot(\cos2x)=\int_{-\pi}^\pi \cos x\cos2x\,dx=0$$ (exercise: explain why the integral is zero), so $\cos x$ and $\cos2x$ are orthogonal with respect to this dot product.

Comments

• In this context, the "dot product" is usually referred to as an inner product and written $\langle f,g\rangle$ instead of $f\cdot g$.
• It is possible to define different inner products for the same space of functions; $\cos x$ and $\cos2x$ might not be orthogonal with respect to a different inner product.
• I doubt that you can visualise orthogonality of functions by means of any simple picture. However, if you continue to study Fourier series, you will soon see why orthogonality of functions is important.

Hope this helps!!

Answer 3

The concept that connects the two notions of orthogonality is an inner product. I'll explain what an inner product is, what it means for orthogonality, and how this more-abstract version of orthogonality relates to the one you're familiar with.

To give a technically correct definition of an inner product I would need to define a vector space, but for this problem that might be overkill. Roughly speaking, a vector space is some collection whose elements can be added, subtracted, and multiplied by real numbers. The set of real-valued functions on $\mathbb R$, for instance, is a vector space, since we know how to add, subtract, and scale real-valued functions.

Note: from now on, when I say vector I'll mean an element of a vector space. So a function is a vector in the vector space of real-valued functions.

An inner product takes in two vectors and returns a scalar; in addition, it must satisfy the following axioms. Think of it as a way to multiply two vectors and return a scalar.

1. $\langle ax_1+bx_2,y\rangle = a\langle x_1,y\rangle + b\langle x_2,y\rangle$ (linear in first variable [by the next axiom, also linear in the second variable])
2. $\langle x,y\rangle=\langle y,x\rangle$ (symmetric)
3. $\langle x,x\rangle = 0$ if and only if $x=0$ (positive-definite)

You are already familiar with one type of inner product: the dot product, which is an inner product on the vector space of row vectors. We say that two row vectors $\vec a$ and $\vec b$ are orthogonal if $\vec a\cdot\vec b=0$. This suggests that a way to make sense of orthogonality in general vector spaces:

Two vectors $v$ and $w$ are orthogonal if $\langle v,w\rangle=0$.

Fourier series start to enter the picture when we look at the vector space of continuous, real-valued functions defined on $[-\pi,\pi]$, say. We can make an inner product on this space by defining $$ \langle f,g\rangle = \int_{-\pi}^{\pi}f(x)g(x)\,dx. $$ (Do you believe that this is an inner product?) Under this inner product, $\sin(x)$ and $\cos(x)$ are orthogonal functions. All I mean is that $$ \int_{-\pi}^\pi \cos x\sin x\,dx = 0, $$ which is true because $\cos x\sin x = \frac12 \sin 2x$. It is possible to show more generally that $$ \int_{-\pi}^\pi \cos(nx)\cos(mx)\,dx = \int_{-\pi}^\pi \sin(nx)\sin(mx)\,dx = 0 $$ if $n\neq m$, and $$ \int_{-\pi}^\pi \sin(nx)\cos(mx)\,dx = 0 $$ always. ($m$ and $n$ are integers.) Here's another example: the functions $1$ and $x$ are orthogonal. So are the functions $x^n$ and $x^m$ whenever $n+m$ is odd (and $n$ and $m$ are nonnegative). Why?