Why is the definition of inner product the way it is?

Question

I have recently started looking at inner products. However, I'm still struggling to intuitively understand the definition.

I am aware that in most cases we generalize concepts because they are useful to do certain things. For example, once we realize that properties like convergence and continuity depend on the notion of distance it seems useful to generalize thr concept. Similarly, norms generalizes the idea of magnitude.

In order to generalize something we need to figure out the defining properties of the concept we want to generalize. For example, for the distance between two points:

(i) It is non-negative and only $0$ if the points are the same.

(ii) It is symmetric.

(iii) The shortest path between two points is a line between them.

This is basically the definition of a metric in words rather than using formulas

However, I can't seem to make sense of the definiton of inner products. I know that the algebraic definition of the n-dimensional scalar product

$$ a \cdot b = \sum_{i=1}^n a_1 b_1+\cdots+a_n b_n$$

originally comes from quaternion multiplication and that there is also a geometric definition using the law of cosines, i.e.

$$ a \cdot b = \|a\|\|b\| \cos(\theta)$$ where $\theta<180^\circ$ is the angle between the vectors. This means that it can be used to determine if two vectors are orthogonal which seems to be the reason why one would like to generalize this concept after all (see for example here or here.

The (real) inner product is defined as

(1) $\langle x+y,z\rangle=\langle x,z\rangle+\langle y,z \rangle$

(2) $\langle \alpha x,y\rangle=\alpha\langle x,y\rangle$

(3) $\langle x,y\rangle=\langle y,x\rangle$

(4) $\langle x,x\rangle\geq 0$, $\langle x,x\rangle=0 \iff x=0$

The properties (2) and (3) seem intuitive to me. (3) only says that the relation is symmetric, i.e. if x is orthogonal to y, then y is also orthogonal to x. (2) says that if two vectors are orthogonal to each other, then scaling one of them does not change this fact. However, I can't see the intuition for (1) and (4).

Can someone please shed some light on this?

Edit: By looking at all the answers it became clear that the scalar product should be seen as a scaled version of the scalar projection of $a$ onto $b$.

Answer 1

Hopefully the following provides some useful perspective:

Think of the inner product as a 'scaled' projection.

Consider $a \cdot b$, and suppose that $\|b\| = 1$. We can write $a= \beta b + w$ where $w \bot b$ and so $a \cdot b = \beta$. In particular (as long as $\|b\|=1$) the quantity $a \cdot b$ gives the projection of $a$ onto the line spanned by $b$.

It is reasonably intuitive that the projection of $a_1+a_2$ should equal to the sum of the projections of the $a_k$, so (1) is reasonable.

From (2) we see that if $a=0$ then $a \cdot a = 0$, and if $a \neq 0$ then we expect $a \cdot a = \|a\| (a \cdot {a \over \|a\|}) = \|a\|^2 >0$.

Answer 2

In the simplest possible terms, the inner product is essentially the "dot product" of an arbitrary vector space. In fact, for a finite-dimensional real vector space (e.g. $\Bbb{R}^n$), the inner product is the dot product. The reason for distinguishing "inner product" as opposed to simply saying "dot product" is that the elementary definition of the dot product...

$$a\cdot b=\sum_n a_nb_n$$

...doesn't always work for an arbitrary vector space. For example, consider the set of all functions $\Bbb{R}\to\Bbb{R}$ with pointwise addition and scalar multiplication defined by $(af)(x)=af(x)$ ($a$ is a scalar, $f$ is a function). This is an infinite-dimensional vector space. Now, suppose that you wanted to find the dot product of two functions $f$ and $g$. Using the formula for the "dot product," you would have...

$$f\cdot g=\sum_{x\in\Bbb{R}}f(x)g(x)$$

Already you see a problem. To calculate the "dot product," you need to take the sum over the set of all real numbers. This is not something that can be done with the basic techniques of analysis, and, in general, is not what we mean when we say "inner product."$^1$ Instead, consider that the "dot product" of two functions is...

$$f\cdot g=\int_{-\infty}^\infty f(x)g(x)\ dx$$

This is an example of an inner product. Of course, not all vector spaces are function spaces, so we need a more general definition that captures the important properties of the "dot product." If you look at the properties shared by the dot product and the "generalised dot product" as defined above, a pattern starts to emerge. If you consider all possible generalisations of the dot product and ask "what does this need?" you end up with the four properties you have listed.

$^1$ There are ways to evaluate such sums, but I've never seen them brought up in the context of vector spaces or discussions of linear algebra.

Answer 3

Here's how you can motivate the coordinate definition of the dot product in $\Bbb R^2$ (for starters) by wanting $x\cdot y = 0$ when $x$ and $y$ are orthogonal. Recall from basic high school geometry that orthogonal lines have slopes that are negative reciprocals of one another (you can get this from basics of similar triangles). So, when $x_1$ and $y_2$ are nonzero (i.e., when the lines are neither horizontal nor vertical), orthogonality is equivalent to $$\frac{x_2}{x_1} = -\frac{y_1}{y_2},$$ which in turn yields $x_1y_1+x_2y_2 = 0$. (And, of course, this formula works fine in the horizontal/vertical case.) Now you get bilinearity and all the rest of the properties, and this shows that $\sum x_iy_i$ is an interesting quantity to study ... :)

Answer 4

To understand the beauty of the inner product (put the intuition aside for now), besides describing the concept as a generalization the OP should know that it is an abstraction, one that 'takes care of' both distance and angles at the same time. I may be biased, but this is a sentence from wikipedia
that 'knocks my socks off',

A Hilbert space $H$ is a real or complex inner product space that is also a complete metric space with respect to the distance function induced by the inner product.

The OP in encouraged to read the history of the how it came about and further developments in that article.

Please indulge me as I pay tribute...

John von Neumann coined the term abstract Hilbert space ...
giving the first complete and axiomatic treatment of them.
-- 1929 --