inner product of a and b

Question

$\langle u,v \rangle = u^Tv$

I am wondering if this only true for column vector, or can this be extended to the matrix case?

Answer 1

It is important to be aware that there are many kinds of inner products. All you need for an inner product (loosely speaking) is the following:

1. Symmetry: $\langle x,y \rangle = \langle y,x \rangle$

2. Positivity: $\langle x,x\rangle > 0$ if $x \neq 0$

3. Linearity: $\langle x+y ,z \rangle = \langle x,z \rangle + \langle y,z \rangle$, and $\langle ax,y \rangle = a\langle x,y \rangle$

That is all!

And each of $x$ and $y$ are called vectors.

Now, in our standard geometry (Euclidean geometry), the inner product of vectors (where we define a vector to be a set of elements that look like: $x = [x_1,x_2,...,x_n]^T$, then yes, the most natural inner product for us to use is the dot product, which is as your write:

$\langle u,v \rangle = u^Tv$

Clearly this satisfies the above rules. Now why this particular form? (1) There is a nice geometrical intuition (cosine product rule) with this sort of expansion and (2) Simple to calculate.

Now if your default vectors were transposed (i.e. row -> column) then naturally it would need to be exapnded as $uv^T$.

Also I mentioned before you need to define how your vectors look. Because technically speaking a matrix can be considered to be a vector object, and so can a function etc... (these are all additive linear objects defined wrt a field etc...).

Now since a matrix can be considered to be a vector-object, it is possible to define an inner product over this also. But it won't look like $u^Tv$ since it is a different object. For real square matrices we usually define the following:

$\langle A,B \rangle = \text{tr}(AB^T)$

And I'm sure there are other definitions to the matrix inner product (because as you can see what constitutes an inner product is very flexible)