Given a tensor of data with the shape $(a, b, c)$, I want to aggregate this tensor such that I have a shape of $(a,)$. Now my question is, does it matter whether I first take the mean across dimension $b$ and then $c$, or first $c$ and then $b$, or if I first reshape the tensor into $(a, b \cdot c)$ and then take the mean across the 2nd dimension.
I went through a toy example with 2 dimensions. My tensor:
\begin{bmatrix} a & b \\ c & d \\ e & f \end{bmatrix}
First, the average across the first dimension:
$\frac{1}{3} \left(\frac{a+b}{2} + \frac{c+d}{2} + \frac{e+f}{2}\right) = \left(\frac{a+b+c+d+e+f}{6}\right)$
Average across the second dimension:
$\frac{1}{2} \left(\frac{a+c+e}{3} + \frac{b+d+f}{3}\right) = \left(\frac{a+b+c+d+e+f}{6}\right)$
Both appear to be equal, as well as equal to first reshaping the data and then averaging. Is this generally the case, or just for this toy example?
Let $X$ be any set partitioned into $n \geq 1$ equal cardinality parts $$ X = X_1 \sqcup \cdots \sqcup X_n,$$ each of size $k = |X| / n$. Let $m_1, \ldots, m_n$ be the means $$ m_i = \frac{1}{k}\sum_{x \in X_i} x,$$ then we have that $$ \frac{1}{n}(m_1 + \cdots + m_m) = \frac{1}{nk}\sum_{x \in X} x,$$ which is the mean of $X$ since $nk = |X|$.
You should be able to iterate this argument to show that if you repeatedly perform operations like this, the total mean at the end will be correct. The key thing is that the partitions $X_i$ are of equal size (which will always happen for your tensors).