Tensor product in multilinear algebra

Question

In the book by Halmos ($FDVS$) the tensor product of two vector spaces U and V is defined as the dual of the vector space of all the bilinear forms on the direct sum of U and V. Is there a generalised form of this for the direct sums of more than two vector spaces? Is there a relation between the space of all multilinear forms on the direct sum of $V_1$,$V_2$,$V_3$,...,$V_k$ with their tensor product.

Please explain without invoking other algebraic objects such as modules,rings etc and by using the concepts regarding vector spaces only (as the book assumes no such background either, it is unlikely that any reader of that book will benefit from such an exposition). Everywhere I searched, I found the explanation in terms of those concepts only and being unfamiliar to those I couldn't get them at all. Thanks in advance.

Answer 1

What is a bilinear form? Naively, it is a certain kind of set-function $\phi\colon V\times W\to \Bbbk$ (where $\Bbbk$ is the base field over which our vector spaces are defined). More cleanly, however, one should think of a bilinear form as an element of the vector space $\DeclareMathOperator{\Hom}{Hom}\Hom_\Bbbk(V,\Hom_\Bbbk (W,\Bbbk))$ where $\Hom_\Bbbk$ is the space of $\Bbbk$-linear (i.e. linear with respect to the scalars in the field $\Bbbk$) maps between vector spaces.

This expresses the fact that evaluating a bilinear from $\phi(v,w)$ in the first variable (plugging in something for $v$) should give a function $\phi(v,-)\colon W\to\Bbbk$ that is linear. So more concisely, we can say that a bilinear form is a linear map in $\Hom_\Bbbk(V,\Hom_\Bbbk(W,\Bbbk))=\Hom_\Bbbk(V,W^*)$, i.e. a linear map from $V$ to the dual of $W$.

What is the tensor product? The tensor product $V\otimes W$ is a gadget which aims to represent bilinear maps as linear maps on a domain which depends naturally on $V$ and $W$. In other words, the tensor product $V\otimes W$ should satisfy the property that $\Hom_\Bbbk(V\otimes W,\Bbbk)\cong\Hom_\Bbbk(V,\Hom(W,\Bbbk))$, i.e. that $(V\otimes W)^*\cong\Hom_\Bbbk(V,W^*)$.

Now, recalling that a vector space $Z$ embeds in its double dual $Z^{**}$, we obtain $V\otimes W\hookrightarrow (V\otimes W)^{**}\cong\Hom_k(V,W^*)^*$. It follows that the tensor product is always contained in the dual of bilinear forms. When $V$ and $W$ are finite-dimensional, then $W^*$ is finite-dimensional, so $\Hom_\Bbbk(V,W^*)\cong (V\otimes W)^*$ is finite-dimensional, so $\Hom_\Bbbk(V,W^*)^*\cong (V\otimes W)^{**}$ is and thus is isomorphic to $V\otimes W$.

To generalize this to multi-linear forms, note that a multilinear form $\phi\colon V_1\times V_2\times\dots\times V_n\to\Bbbk$ should be thought of as an element of the vector space $\Hom_\Bbbk(V_1,\Hom_\Bbbk(V_2,\dots,\Hom_\Bbbk(V_n,\Bbbk))\dots)=\Hom_\Bbbk(V_1,\Hom_\Bbbk(V_2,\dots,\Hom_\Bbbk(V_{n-1},V_n^*))\dots)$. This should be naturally isomorphic to $\Hom_\Bbbk(V_1\otimes V_2\otimes\dots\otimes V_n,\Bbbk)=(V_1\otimes\dots\otimes V_n)^*$, so again $(V_1\otimes\dots\otimes V_n)^{**}\cong\Hom_\Bbbk(V_1,\Hom_\Bbbk(V_2,\dots,\Hom_\Bbbk(V_{n-1},V_n^*)\dots)^*$, i.e. again the double dualr of the multi-variate tensor product should be naturally isomorphic to the dual of the space of multi-linear forms. When the latter is finite-dimensional (if all the $V_i$ are finite-dimensional), then so is its dual, hence so is the double dual of the tensor product.

(There is nothing weird about the fact that the tensor product embeds in the dual of multi-linear forms. Taking a basis vector $v_1\otimes v_2\dots\otimes v_n$, by definition one can evaluate a multi-linear function $\phi\colon V_1\times\dots\times V_n\to\Bbbk$ at it by doing $\phi(v_1,v_2,\dots,v_n)$).

---

To address the further questions in the comments:

1. The tensor product $V\otimes W$ of $n$-dimensional $V$ with $m$-dimensional $W$ is (according to the above) an $n\cdot m$-dimensional vector space. More precisely, however, the tensor product $V\otimes W$ is an $n\cdot m$-dimensional vector space equipped with additional structure. This additional structure consists of a single piece of data: a designated isomorphism of the dual of the tensor product $(V\otimes W)^*$ with $\Hom_\Bbbk(V,W^*)$. This means that it does not actually matter which $n\cdot m$-dimensional vector space you consider to be the tensor product, what matters is the additional structure you put on it (or rather on its dual). So in fact every $n\cdot m$-dimensional space can be given the structure of being the tensor product $V\otimes W$, and then there are isomorphisms that preserve this structure, so up to isomorphism there is a unique tensor product.

2. Compare to how an inner product space is a vector space $V$ with a positive-definite symmetric bilinear form $\phi(-,-)\colon V\times V\to\Bbbk$. Being a bilinear form means that this is really a map in $\Hom_\Bbbk(V,V^*)$, and the other properties (positive-definiteness, symmetry) are properties of that map. Choosing different inner products gives different inner product structures (e.g. a positive weighted dot product versus the usual dot product on $\mathbb R^n$). Nevertheless, all inner product spaces of the same dimension are not only isomorphic as vector spaces, but the isomorphisms can be chosen to preserve the extra structure of the inner product, i.e. they are isometric. So again, up to isomorphism, there is only one inner product space of any particular dimension.

Answer 2

If it's tensor products of more than two vector spaces, and not direct sums, yes, it's exactly the same characterization. (Usually a bilinear form is defined as a map $U \times V \to F$, where $F$ is the underlying field, which is linear in both arguments. But $U \times V$ is taken as a set, without the structure of a direct sum.)

That is, you can define the tensor product $V_{1} \otimes V_{2} \otimes \dots \otimes V_{k}$ as the dual of the space of all multilinear forms on $V_{1}, V_{2}, \dots , V_{k}$. The reason is that you want a universal property to hold so that there is an isomorphism of vector spaces between

• the space of multilinear forms, and
• the space $T^{\star}$ of linear forms on the tensor product $T$.

So take the dual, and note that in the finite-dimensional case $T^{\star \star} \cong T$.

Answer 3

First of all: yes: just replace "bilinear" with "multilinear" and you have a definition that works for finite-dimensional vector spaces.

Second: as Halmos also says right there, this is not the correct definition for vector spaces in general. The reason is that the correct definition is actually indirect: it says that the dual of the tensor product is isomorphic to the space of multilinear forms on the direct sum. For FDVS's, this is equivalent to what he wrote, because the double dual of an FDVS is itself, but for infinite-dimensional spaces, this is not so.

Of course, for FDVS's you have the unsatisfying definition that "the tensor product of a $m$-dimensional and an $n$-dimensional space is an $mn$-dimensional space", since they are all determined completely by their dimensions. But that's not what's important in tensor products: what matters is the relationships that exist between vector spaces. And the relevant relationship for tensor products is:

For any vector spaces $V, W$ and $U$, there is a natural correspondence (in a sense that can be made precise using ideas you asked not to use) between linear maps $V \otimes W \to U$ and bilinear maps on $V \times W$ valued in $U$.

If we used the 1-dimensional space for $U$, we would recover the definition of $V \otimes W$ as the dual of the space of bilinear forms, but the above text is the real definition.

Answer 4

(As in my comment at the top...) Sincerely and seriously, apart from whatever short-term adaptations may be necessary or desirable... As @RyanReich's answer and comments suggest, an "elementary" "definition" of tensor product will leave a person mystified, for one thing. For another, it's completely unclear why we need such a thing. (Be thankful you aren't studying the late-nineteenth-century version of linear and multi-linear algebra, with "upper and lower multi-indices", thus spawning "covectors" $v^i$, which somehow were different from "vectors" $v_i$. "Different"? In what way? Well, supposedly, in the "rules"... Nevermind.)

The long-term advantageous viewpoint, which is much better for far more reasons than just understanding tensor products per se, has two parts. The first is about the transition from "definition" in the sense of "construction" (usually in set-theoretic terms), to "characterization" in the sense of how the thing is intended/required to interact with other things. Yes, this inverts the lower-level textbook-y approach of the last several decades, in which we (somewhat disingenuously!) give a "definition" at the very beginning ... not admitting that people went through hosts of examples and many trial definition before hitting on the best abstraction ... and then "proving" the (desired/required) properties in an exaggeratedly formal (but stylish) style. In particular, for tensor products, while one has been conditioned to expect "a construction", in truth that's not what we truly want: we want to know what the point is, what's going to be accomplished. But a certain overly-formal textbook version of mathematics has scant way to say this, except for... (drum-roll)...

"Category theory". Informal/naive category theory is a wonderful thing, in the same way that informal/naive set theory is a wonderful thing, as it gives a language and family of concepts that enrich us, that broaden the possibilities, show commonalities, etc. One must be cautious about the more "formal/axiomatic" versions, however, at least that they do tend to be "X for its own sake" or "axiomatizations of tangible things..." E.g., 100 years ago it was indeed interesting to hear from Kuratowski, Wiener, and others, that "an ordered pair" $(a,b)$ could be modeled "in set theory" (wherein "everything's a set") as $\{\{a\}, \{a,b\}\}$. But, srsly, folks, it is the characterization of "ordered pairs", namely, that $(a,b)=(a',b')$ if an only if $a=a'$ and $b=b'$ that we care about. That is, we care about the interaction of "an ordered pair" with the individuals "in" it, and with other ordered pairs, not the construction... although it's nice to know that clever people can model ordered pairs as sets.

Second, ... and here's a not-so-elementary point... a good "external" characterization of a thing often so-completely characterizes it that, by the very characterization (if done right), any two things $X,Y$ that do the job have a (unique) isomorphism between them (that also fits perfectly with whatever job is required). At a point where one distinguishes "equality" from "isomorphism" (which is sometimes apt, sometimes not), it is noteworthy that this does not say $X=Y$, but only that they are (uniquely) isomorphic. In truth, mostly this is the most one can reasonably demand. Often, this part follows _prior_to_ any construction! By the characterization! To repeat: although this is somewhat alien to elementary mathematics (typical undergrad?), the construction is not of interest, the interaction (as external characterization) is the point.

For example, with more-familiar objects: a quotient group $Q=G/N$ is a group $Q$ and (quotient) homomorphism $q:G\to Q$ such that any group hom $F:G\to H$ with $N$ inside its kernel "factors through $Q$, in the sense that there is $f:Q\to H$ such that $F=f\circ q$. Drawing a picture is nice, here, too.

What is an "indeterminate" $x$, as in the case of a polynomial ring $k[x]$ over a field $k$? We think of "variable" things, quite reasonable, but, seriously, what can this mean? It means that, for example, given a commutative ring $R$ with $k$ inside it, and given $a\in R$, there is a unique ("$k$-algebra") homomorphism $k[x]\to R$ sending $x$ to $a$.

For tensor products: a tensor product $V\otimes W$ of two $k$-vectorspaces is a $k$-vectorspace and bilinear $b:V\times W\to V\otimes W$ such that, for every bilinear $\beta:V\times W\to X$ for $k$-vectorspace $X$, there is a unique $k$-linear map $B:V\otimes W\to X$ through which $\beta$ factors, by $\beta=B\circ b$. From this characterization (not construction) all the relevant things can be proven. Does such a thing exist? Well, yes, and there are various (not so useful, really) constructions.

Part of the issue is that we are often given "the definition" of tensor products without any persuasive argument for caring about such things... and, then, the fact that they are somewhat more sophisticated things makes it doubly confusing.

So, that was the five-minute version of something. :)