Difference between dot product, inner product, cross product, outer product. And what are their symbols?

Question

When I was learning machine learning, I often encounter these terms, yet I don't the difference and relationship between them. And what are the symbols for them? Can someone help me?

Answer 1

The inner product is denoted $\langle x,\,y\rangle$ (or occasionally $(x,\,y)$) or $\langle x|y\rangle$. It mustn't be cofused with an outer product, which is basically a square matrix. You may see it denoted as $\sum_{ij}|i\rangle\langle j|$.

For your purposes, all these products will be on a finite-dimensional vector space, and with respect to some orthonormal basis the inner product will just be a dot product $x\cdot y=\sum_ix_i^\ast y_i$ (you only need the ${}^\ast$ if the space is complex). In a space of uncountable dimension, the $\sum$ in the dot (outer) product becomes a single (double) integral.

The cross product $x\times y$ almost certainly refers to something exclusive to $3$ dimensions, but sometimes it's something related but different.