I am not a mathematician by any means, so please be clement if my question happens to be naïve.
In many textbooks, matrices make their first appearance in the context of a discussion of linear transformations of vectors. Then the author usually proceeds to consider changes of basis, deduce formulae like $T^{-1}AT$, and so on. So far, so good, except for the fact the term «matrix» actually disguises the tensor nature of linear transformation.
Matrix is just a representation of tensor in this context. But what is important here is that despite their somewhat abstract nature, tensors correspond to physical values. However, the notion of covariance has no meaning when one talks about matrices. That makes it possible to consider matrices as in some sense as more general objects than tensors (speaking simplistically «every tensor is a matrix, but not every matrix is a tensor»; yes, tensors can be of any rank, but let's talk about 2D case only).
Many mathematical objects are intimately connected to the properties of physical objects. Of course, by definition, they are generalizations and abstractions, but this intimate touch with reality persists nonetheless. Here are a few examples of such connections: continuity — topology; counting — ordinary numbers and fractions; rotation — complex numbers and quaternions; the rate of change — derivative; linearity — tensors (vectors) and so on. I would not go too far and say that every mathematical object should have some physical meaning, even in principle. It is maybe that matrices are an example of such «pure mathematical machinery.» But, if so, how can it be that they appear in many physical equations (well, they pervade all mathematical physics!)? It looks like, for example, Pauli of Dirac matrices should have some meaning. And this is my question: can matrices (not the objects they represent) be associated with any property of the world.
There are essentially two ways matrices arise from physics.
On one hand, physics is generally described through differential equations, and quantities are often defined through derivatives of other quantities. Now a derivative in essence means locally approximating a function by a linear function. And linear functions are represented by matrices.
Also the second derivative of scalar quantities is often used, which means local approximation by a quadratic form. And again, quadratic forms are described by matrices.
On the other hand, in modern physics many of the fundamental laws are derived from symmetries. Those symmetries form a group, and representation theory tells us that any group is isomorphic to a subgroup of the general linear group of an appropriate vector space. The general linear group consists of all invertible linear functions of that vector space to itself. Again, when the vector space is finite, the linear functions can be represented as matrices.