I'm trying to get an idea of the amount of information that is "stored" in an "unlabelled" matrix.
I assume that the vector $(x,y,z)$ contains more information than the set $\{x,y,z\}$. But purposely allowing permutation within the index, they should be equal in information content. (Right?)
Now how about an $m\times n$ matrix $A$ containing $m\cdot n$ reals? Will allowing permutations within the indices (i.e. unlabelling) also reduce its information content to that of the set $\{A_{11}, A_{12}, \ldots, A_{m,n-1}, A_{mn}\}$? (Or does the "combinatorial structure" - I just made that up - of the indices matter?)
And, but that may come later, will the unlabelling of the dimensions (i.e. purposely allowing transposition) affect the information content?
What's the best way to look at this?
Because the above might be somewhat poorly phrased, let me clarify where I'm coming from. The normal-form game in game theory is a matrix. However, it is understood that any analysis done on that matrix is invariant with respect to the "position" of rows and columns. E.g., you may represent your Defect strategy by the left column or by the right column. All such permutations are allowed and shouldn't change your analysis. Therefore, they lack information. So, how much information is there in the whole class of such permutated matrices? Oh boy, this isn't getting much clearer is it?
I'm thinking along the lines of something like $$\frac{\text{Information content equivalent of a tuple of }m\cdot n\text{ reals}}{m!n!}.$$
I'm not sure I understand the question, but I'll give it a shot, especially the "how to think about this" part.
Let's pretend that all the tuple and matrix values are real numbers, for simplicity. A tuple $(x,y,z)$ is a mapping from {1,2,3} to the reals, or it's an element of $R^3$. But {x,y,z} is just a set.
Think of the $x$, $y$, $z$ values as the heights of bars on a bar chart. The tuple $(x,y,z)$ gives you a meaningful bar chart, but with $\{x,y,z\}$ you know the heights of the bars, but not their order -- significantly less information.
From $\{x,y,z\}$, you can get aggregate information like mean, max, min, variance, but not much more.
The same reasoning applies to matrices.
Now that I read this, it doesn't seem very helpful, actually. Shoganai.