I'm an undergraduate student who already finished a linear algebra and vector calculus class. I'm wondering if matrix calculus is a specific case of tensor calculus and if theorems and results from matrix calculus can be deduced using tensor calculus.
Finally, is it necessary to learn matrix calculus first to understand tensor calculus?
On
Think of vectors for a moment in the physics sense of a basepoint and an offset from that basepoint. If I want to move a vector in $\mathbb{R}^n$ from one basepoint $p$ to another $q$, it's easy: I just apply a linear translation. Suppose instead that I'm on longer working in $\mathbb{R}^n$, but rather on some more complicated smooth manifold $X$. (If you're not familiar with manifolds, think of something like the unit sphere in $\mathbb{R}^n$.) At each point $p\in X$, I have a convenient set of vectors to work with: the tangent space $T_p X$ to $X$ at $p$, which I can think of as physics-style-vectors with basepoint $p$ that are tangent to $X$ at $p$. The problem, though, is that there's no obvious way to go from vectors in $T_p X$ to some other $T_q X$. The tangent space moves in complicated ways as $p$ varies, and one is not just a linear transformation of the other anymore. Furthermore, because the translation is nonlinear, complications arise when I compute derivatives and other higher-order terms. Tensor calculus is a way of managing all that bookkeeping. In particular, quantities that are physically meaningful should be have in the expected way under coordinate transformations, and so it's useful to check that the bookkeeping works out.
Of course, I'm skipping over a lot of details here. The more precise definition is that tensor is a section of a particular bundle over a manifold; that condition can be unrolled into a bunch of explicit coordinate transformation properties, but it's usually not enlightening or useful to do so. Tensor calculus can refer specifically to the types of constructions often seen in Riemannian geometry or in physics (particularly relativity). There's also the matter of tensor products in linear algebra or, more generally, modules. (These are both often covered in classes on commutative algebra, rather than linear algebra.) This is more directly related to the idea of multilinearity, and it's analogous to the idea of tensors as a higher-dimensional version of matrices. And, in fact, matrix algebra can be constructed as a specific case of tensor algebra in this latter sense. It's just not usually done that way because matrices are simpler, have particular properties that are worth studying on their own, and are more common in both theory and application.
There are three important cases where matrices are employed
As mechanism of determine basis changes inside a finite vector space.
As a linear transformation among vector spaces.
As a pairing for a bilinear form and pairing of a quadratic form.
The first is to study behavior of the components of a vector in case that one wants to describe them when the space "suffers" of different choice of a basis.
The second is used to study the transformation's properties between vector spaces, stressing the geometric' like ones.
The third as maps $V\times V\to\mathbb R$ via $(v,w)\mapsto v^{\top}Qw$, where $Q$ is a square matrix gives big transparency to the meaning of being bilinear, and this pairing generalizes the notion of inner products, and the last sub-case, $V\to\mathbb R$ mapping via $v\mapsto v^{\top}Qv$, is for the handling of quadratic functions on $V$ is also bilinear.
The modern vision that surrounds the concepts and methods of the tensors can be summarized as the study of the multilinear transformations among vector spaces.
Matrices are a type of tensors dubbed rank two tensors.