Question concerning tensors

Question

As some of you may have seen from my previous question(s), I am working through Spivak's Calc on Manifolds and this happens to be the first time I've been introduced to tensors formally, though I have come into contact with specific instances of them in certain classes in school (fluid mechanics, solid mechanics, electromagnetism). I'm asking if anyone has a specific technique for understanding these objects. The following is my (imprecise) interpretation, please correct me if I'm wrong to think this way and as usual, any insight is greatly appreciated.

So a 0 order tensor is a scalar, a tensor of order 1 is a vector. My understanding of a 2nd order tensor is that, since each element requires two indices to specify, a 2nd order tensor may be represented as a matrix, when "placed" in any coordinate system of course. Now, if we continue this pattern, it seems to me that in order to represent a 3rd order tensor, one must actually construct a geometric "object" somewhat analogous to a cube, with each face being a matrix. Is this interpretation correct?

As a side note, if the above is correct, it does seem that such objects would inevitably have been discovered at some point. Why confine the quantitative relationships between mathematical objects to a point, line or plane? It seems this is a generalization which naturally should arise.

Answer 1

I know of no algebra in which 3d arrays of numbers can be manipulated with the same ease as matrices. Part of that has to do with how any linear map from one space to another space can be represented with a matrix. You can chain such maps together sensibly (and really, only in one way up to the order of how you compose those operations together), whereas chaining general tensors together has considerably more freedom: when you're chaining operations together, on which arguments do they act? And so on.

Further, I would object to considering matrices to be "geometrical" in any such sense. Yes, you can write them down as a 2d array, but that arrangement of entries in the matrix doesn't have anything to do with the actual geometry of the underlying manifold or vector space.

Answer 2

I wrote a lot, due to boredom and the general hope that some of it is helpful. Much will probably be familiar to you.

First: In order to work with tensors or to do calculus on manifolds at all, it's very important to start making the distinction between vectors and covectors (or "dual vectors" or whatever). Briefly, if $V$ is a (finite dimensional, real) vector space, there's an associated (real) vector space $V^*$ which is the set of linear maps $V \to \mathbb{R}$. $V^*$ is called the dual to $V$, and its elements are called covectors or dual vectors.

When $V = \mathbb{R}^n$ as in college linear algebra courses, we often make the identification of $V$ with column vectors, and $V^*$ with row vectors. We have an easy way of taking a column vector and making a row vector or vice versa (taking the transpose), so often we don't distinguish between the two and just call everything a "vector". But on a manifold, there are many different possible choices of coordinates making the tangent spaces look like $\mathbb{R}^n$ in different ways, which means that there's no "natural" way to take a "transpose" of a tangent vector. This means that it's important to think of tangent vectors and tangent covectors as very different things; which means back in linear algebra world, we need to differentiate between $V$ and $V^*$.

Said another way, there's two different kinds of "one-tensors" on a vector space $V$: what's called a $(0,1)$-tensor, which is an ordinary vector, an element of $V$, and what's called a $(1,0)$-tensor, which is a covector, an element of $V^*$. It's the right intuition to think of $(0,1)$-tensors as column vectors and $(1,0)$-tensors as row vectors, but you should remember that "transpose" isn't meaningful in this generality (and won't be when you start talking about tangent spaces to smooth manifolds).

So in general it's confusing to talk about "$n$-tensors"; instead the correct notion is that of a $(k,l)$-tensor, which is a multilinear map $$ T : V \times \cdots \times V \times V^* \times \cdots \times V^* \to \mathbb{R}, $$ where there are $k$ copies of $V$ and $l$ copies of $V^*$.

A matrix $A$ is a $(1,1)$-tensor, because it eats a row vector (an element of $V^*$) and a column vector (an element of $V$) and produces a number. Differential geometers will write the element of a matrix $A$ in the $i$-th row and $j$-th column as $A^j_i$; the idea of writing the $i$ as a subscript, and the $j$ as a superscript, is to explicitly indicate that the matrix is a $(1,1)$-tensor.

A good example of a $(2,0)$-tensor on a vector space $V$ is an inner product, since an inner product is suppose to eat two vectors and tell you something about the angle between them and their lengths and so forth. It's misleading to picture an inner product as a matrix -- you want to say that the inner product given by the matrix $A$ is the map taking the vectors $u, v$ to $u^T A v$, but without coordinates we can't make sense of $u^T$, which means that we won't be able to make sense of $u^T$ in a coherent way when we're on the tangent space to a manifold (where there are many choices of coordinates).

So if you wanted to be able to take tangent vectors (at the same point) to your manifold $M$ and do inner-product things to them like measure the angle between them, you'd want for each tangent space $T_pM$ a $(2, 0)$-tensor on that space -- known as a $(2,0)$-tensor field on $M$ -- with some nice properties that make it an inner product and "vary smoothly". A $(1,1)$-tensor field just wouldn't do the job! (But I digress... I don't want to be discussing tensor fields.)

This is all a preamble to saying that I think the intuition "a $3$-tensor is like a cube of numbers" is not a particularly helpful way to think about things, because (1) you can't write a cube of numbers down on paper very easily and (2) it obscures that there's a fundamental difference between "row-like" things and "column-like" things, and that each of the "axes" of this "cube of numbers" should really be marked "row-like" or "column-like" to determine whether it acts on column vectors or row vectors. (There really are only these two kinds of vectors we're considering in differential geometry -- no weird "depth vectors" or whatever.)

So if you want to think about tensors in coordinates, with numbers, like you do when you call a $(1,1)$-tensor a matrix, I think the best you can do is to think of a $(k,l)$ tensor $T$ as a list of numbers $$ T^{j_1, \dots, j_k}_{i_1, \dots, i_l} $$ where the $i$'s and $j$'s each run from $1$ to $n$ (where $n = \dim V$). (Just like with writing down the matrix of a linear transformation, actually writing a tensor as such a list of numbers requires first choosing a basis for $V$.) The top indicies run across generalized "rows" and the bottom down generalized "columns", but I'm not sure this really means anything or is a useful way to think. I don't have a coherent system for writing such huge lists/arrays/whatever down on paper. Relatedly, it's also the case that I almost never do write such things down on paper.

How does this list of numbers become a linear transformation? It's just like matricies. Fixing a basis on $V$ (and hence also on $V^*$), we can write elements of $V$ as $(0,1)$-tensors, which are lists of numbers $v_i$, for $1 \leq i \leq n$, and elements of $V^*$ are $(1,0)$-tensors, which are lists of numbers $w^j$. So say we have a $(3,2)$-tensor $T^{j_1, j_2, j_3}_{i_1, i_2}$ and three vectors $u_i, v_i, w_i$ and two covectors $a^i, b^i$; then the tensor $T$ evaluated at $(u, v, w, a, b)$ is the number $$ \sum_{1 \leq j_i, j_2, j_3, i_1, i_2 \leq n} T^{j_1, j_2, j_3}_{i_1, i_2} u_{j_1} u_{j_2} u_{j_3} a^{i_1} b^{i_2} , $$ which is what it looks like -- a mess. Luckily, we don't have to think about tensors this way very often.