I want to adequately define a $m\times n$ "multimatrix" that satisfies these properties:
0.A $m\times n$ multimatrix has $m\times n$ entries just like a normal matrix. It is the positions they occupy that is the difference between a multimatrix and a matrix.
1.A row or column can be in more than one positions simultaneously. The positions of an entry are determined by the row and the column it is on.
2.Different rows or columns can occupy the same position but are still separate.
3.Row and column operations that happen in matrices also happen in multimatrices in the same manner.
I have searched Wikipedia and have not seen any existing structure like this.
For example: $M=\{c_1,c_2\}$. $c_1$ occupies positions 1 and 2 (like an electron that people do not know where its exact position is..). $c_2$ occupies position 2' (can be thought of as stacking $c_2$ on $c_1$.) Now perform the column operation $c'_1\leftarrow c_1+c_2$.
Now $M$ has been transformed to $M'=\{c_1+c_2,c_2\}$. $c'_1 = c_1 + c_2$ occupies positions $1$ and $2$. $c_2$ occupies position 2'.
My problem is how do permutations work in this multimatrix structure? In a normal matrix we can let a permutation group element act on the matrix by permutating rows and columns. How does that work here? Do we need to use finite matrix groups instead of $S_m$ or $S_n$ now?
OK I answer the question myself.
Let's introduce some notations first. $[m]$ is defined as the set of integers $1$ to $m$ for all positive integers $m$. $\tilde{M}_{m\times n}(R)$ is the set of all $m\times n$ multimatrices with entries elements of a commutative ring $R$. Entries of $A\in \tilde{M}_{m\times n}(R)$ are called $a_{ij}$. The set of positions of row $i$ is $p(i)$. The set of positions of column $j$ is $q(j)$.
In essence, the structure of multimatrices $m\times n$ is contained in the structure of matrices $m2^m\times n2^n$. (I do believe this can be written rigidly in a category-theoretical sense but I have not thought about that yet).
Here is how:
Each element $a_{ij}$ of a multimatrix $A\in \tilde{M}_{m\times n}(R)$ for some ring $R$ has row positions a subset of $[m]$ and column positions a subset of $[n]$. It is easy to label subsets of $[m]$ by integers $1$ to $2^m$ and subsets of $[n]$ by integers $1$ to $2^n$. These functions $2^{[m]}\rightarrow [2^m]$ and $2^{[n]}\rightarrow [2^n]$ are denoted by $\sigma_m$ and $\sigma_n$ respectively. Consider a matrix $B=\phi(A)\in M_{m2^m\times n2^n}(R)$ with all but these entries zero:
$b_{2^m(i-1)+\sigma_m(p(i)),2^n(j-1)+\sigma_n(q(j))}=a_{ij}$
Here all linear properties are preserved.
P.S. The problem of constructing an injection from multimatrices to arbitrarily large (though finite) matrices can be divided into two parts:
1.One row/column can occupy multiple positions. In essence rows/columns do not occupy elements of $[m]$/$[n]$ any more but occupy subsets of them (or elements of $2^{[m]}$/$2^{[n]}$).
2.Different rows/columns can occupy the same position which is not allowed in matrices. So we just add more rows and columns so that different rows/columns can not actually occupy the same spot in the matrix $\phi(A)$.
We still use a multimatrix $A$ as opposed to always use the really large $\phi(A)$ since it is more convenient.