Are rank 3 tensors always cubes?

Question

a matrix is $A\in \mathbb{R}^{3\times 3}$. It is symmetric and contains 3 row vectors and 3 column vectors containing elements $a_{i,j}$. It looks like a square and, as long as the two dimensions are of equal order, the matrix is always a square .

a 3-rank tensor is $B\in \mathbb{R}^{3\times 3\times 3}$. Is it symmetric for its 3 dimensions? If we can call its first two ranks row and columns vectors also, what do we call the 3rd-rank vectors? and given that the matrix $A$ was a square, does this make 3-rank tensors cubes?

What is a correct way to visualize a 3-rank tensor in its entirety, element-by-element ($b_{i,j,k}$, etc) as you would visualize $A$ in its entirety by indexing its elements one by one into rows and columns? and how to make the concept of a 3-rank tensor intuitive for someone with a strictly statistical (matrix algebra) background (who doesn't look at vectors as bearing physics connotations like "direction", but are merely containers of elements and nothing more)

Answer 1

If $B$ is rank three tensor in $\mathbb R^3$, one have three $3\times 3$-matrices $$B_{1ij}\quad,\quad B_{2ij}\quad,\quad B_{3ij},$$ allowing the indexes $i,j\in\{1,2,3\}$.

Those are the $27$ components $B_{kij}$ of $B$ arranged in packets or alternatively $$B_{k1j}\quad,\quad B_{k2j}\quad,\quad B_{k3j},$$ and $$B_{ki1}\quad,\quad B_{ki2}\quad,\quad B_{ki3},$$ giving a gran total of nine $3\times3$-matrices.

Answer 2

(Why are you saying that a rank-2 tensor is always symmetric? That's certainly not true. Maybe you are not referring to symmetry in the sense of $A_{ij} = A_{ji}$?)

The choice of notation to write a matrix as rows and columns, or to think of a rank-3 tensor as a cube, is entirely a choice of notation and is not required by the objects anywhere.

When thinking of a matrix as a linear transformation $U \rightarrow V$, its terms are elements of $U^* \otimes V$, where $U^*$ is the dual space to $U$. This is the usual meaning of $U_i^j$. But sometimes you find yourself with a rank-2 tensor $U \otimes V$ or $U^* \otimes V^*$, where neither or both sides are dual spaces, and I find these more natural to think of as 'columns of columns' or 'rows of rows', like this:

$$A_{ij} = \big( (A_{11}, A_{12}, A_{13}), (A_{21}, A_{22}, A_{23}), (A_{31}, A_{32}, A_{133}) \big)$$

This lets one maintain the notation that an inner product is always a contraction of a row with a column.

Anyway, a mental model for a rank-3 tensor could be a cube, or a matrix whose entries are themselves rows or columns, or a column of columns of columns. Whatever you want. Once you're dealing with $>2$ dimensions, it tends to be a lot easier to use https://en.wikipedia.org/wiki/Einstein_notation rather than trying to figure out how to write the object out as a matrix-like thing. If you really want a visualization, though, I suggest a matrix whose components are columns or rows, like this:

$$A_{ij}^k = \begin{pmatrix} \begin{pmatrix} A_{11}^1 & A_{12}^1 \end{pmatrix} & \begin{pmatrix} A_{21}^1 & A_{22}^1 \end{pmatrix} \\ \begin{pmatrix} A_{11}^2 & A_{12}^2 \end{pmatrix} & \begin{pmatrix} A_{21}^2 & A_{22}^2 \end{pmatrix} \\ \end{pmatrix}$$

Easier to write. You have to be careful when multiplying it with anything to keep track of which index is which -- $A_{ij}^k v^i \neq A_{ij}^k v^j$, as the former multiplies the outer dimension but the latter multiplies the inner one.