I'm reading Chapter 5 (Inner Product Spaces) of Linear Algebra Done Wrong. One of the examples of an inner product space was the following:
Let $V$ be the space $\Bbb P_n$ of polynomials of degree at most $n$. Define the inner product by $$ (f,g) = \int_{-1}^1 f(t) \overline{g(t)} dt .$$
My question is: where on earth does that come from? I can see that the basic properties of inner product spaces hold here. What I can't see is why this is a useful definition and how this relates to inner products in $\Bbb C^n$. So how should I think about this definition?
This is a good question to ask. It is hard to give a complete and satisfactory answer, but here are some slightly disjoint facts that may help.
The nice thing about inner product spaces (in particular complete ones) is that our geometric intuition remains a good guide.
Think of it analogous to the standard inner product on $\mathbb{C}^n$. We have $\langle x , y \rangle = \sum_k x_k \overline{y}_k$, and if we replace the index $k$ by $t$ and the summation by the integral we get a similar formula.
Note that if $P$ is positive definite, then $( x , y )_P = \langle x , Py \rangle$ is also an inner product on $\mathbb{C}^n$.
A normed space is an inner product space iff the norm satisfies the polarisation identity (See https://en.wikipedia.org/wiki/Polarization_identity). The point is that it might be easier to think in terms of the corresponding norm, $\|f\|_2 = \sqrt{\int_{-1}^1 |f(t)|^2 dt}$ first, rather than trying to understand the inner product in terms of angles and the like. The norm $\|\cdot \|_2$ often has an interpretation of the 'energy' of a signal $f$.
Since $\mathbb{P}_n$ is finite dimensional and a 'function space', it provides a bridge between the two inner products. If $p \in \mathbb{P}_n$ we can identify $p$ with an element of $\mathbb{C}^{n+1}$, that is, if $p(t) = \sum_k \hat{p}(k) t^k$, then we identify $p$ with $(\hat{p}(0),...,\hat{p}(n))$. Then if $p_1,p_2 \in \mathbb{P}_n$ we can write $\langle p_1, p_2 \rangle = \int_{-1}^1 p_1(t) \overline{p}_2(t) dt = \sum_i \sum_j \hat{p_1}(k) \overline{\hat{p_2}(k)} \int_{-1}^1 t^{i+j} dt = (\hat{p_1}, \hat{p_2} )_T $, where $T$ is a positive definite matrix. (I have skipped over the details of the map $p \mapsto \hat{p}$ here.)
In particular, for elements of $\mathbb{P}_n$ the integral inner product is essentially the same as an inner product on $\mathbb{C}^{n+1}$.