Interpretation of Einstein notation for matrix multiplication

Question

Consider the matrix product $C = AB$ where $A \in \mathbb{R}^{m \times n}, B \in \mathbb{R}^{n \times p}$. The Einstein summation notation for this is $$ c_{ik} = a_{ij}b_{jk}. $$ Is there any example from math, physics, engineering, statistics etc. where each term in the sum $a_{ij}b_{jk}$ has a meaningful interpretation?

Equivalently, I guess: suppose we did not use Einstein summation notation, and instead defined $D \in \mathbb{R}^{m \times n \times p}$ as: $$ D_{ijk} = A_{ij}B_{jk}. $$ Would the elements of $D$ mean anything with respect to $A, B, C$ or the linear transformations represented by these matrices? Obviously $C_{ik} = \sum_{j} D_{ijk}$, but what does this mean?

Or suppose, instead of summing over the second index of $D$ to get $C$, we instead summed over the first index of $D$ to get $E \in \mathbb{R}^{n \times p}$: $$ E_{jk} = \sum_{i} D_{ijk}. $$ Obviously this cannot be represented in Einstein notation. So does this not have meaning as a tensor contraction, and unlikely to have a physical interpretation?

Edit: For the powers of adjacency matrices (eg $C = A^2 = AA$), the terms do have an interpretation. Iff $a_{ij}b_{jk} = 1$ (not using Einstein notation), then there is a path from node $i$ to node $k$ through node $j$.

Answer 1

Define a third order tensor whose components are equal to zero unless all three indices are equal $${\cal H}_{ijk} = \begin{cases} 1 \quad{\rm if}\; i\!=\!j=\!k \\ 0 \quad{\rm otherwise} \\ \end{cases} $$ Then you can use Einstein notation to write $${\cal D}_{ijk} = A_{ip}{\cal H}_{pjs}B_{sk}$$ This tensor is a useful addition to standard matrix algebra.

It can be used to generate a diagonal matrix $A$ from a vector $a$ (using a single-dot product) $$A = \operatorname{Diag}(a) = {\cal H}\cdot a \quad\implies A_{ij} = {\cal H}_{ijk}\,a_k$$ or to create a vector $b$ from the main diagonal of a matrix $B$ (using a double-dot product) $$b = \operatorname{diag}(B) = {\cal H}:B \quad\implies b_{i} = {\cal H}_{ijk}\,B_{jk}$$ or simply as a way to write ${\cal D}$ without resorting to index notation $${\cal D} = A\cdot{\cal H}\cdot B$$

Answer 2

One way to think about this is as if you're taking the scalar/inner/dot product of the vectors $a_i$ and $b_k$ by indexing these two vectors with j. In this way, each entry of the matrix $c_{ik}$ is exactly the inner product in question.