Let $S$ be the set of all $n \times n$ real matrices ($n$ > 1) with entries from a finite set $M$. Can we say that the average value of determinants of matrices in $S$ is always zero?
See that $S$ is closed under row interchange. So, we have matrices with equal determinant but of opposite sign. I think, we have to show that there is a bijective map between the subset of matrices with positive determinant and subset of matrices with negative determinant. Is that enough? Observe that the problem is independent of nature of the entry set $M$.
The idea is correct, but the above formulation is not. There is a bijection between the sets $\{1\}$ and $\{-4\}$, one containing only positve numbers, the other only negative numbers, still the average of all numbers is not $0$.
The correct formulation of the idea would be to try to find a bijective map between the subset of matrices with positive determinant and subset of matrices with negative determinant, where the corresponding matrices have determinants of the same absolute value (but opposite signs).
As you noted, interchanging 2 fixed rows (or columns) is exactly such a map. It changes the sign of the determinant, but keeps the absolute value. Depending on the rigor needed/wanted for the proof, you should think about why this is really a bijection! It's not hard, but sometimes in such proofs that is not so easy, and sometimes what looks like a bijection isn't one.