Let $S\subseteq\mathbb{F}_q^{n\times n}$ be a subset of square matrices over a finite field of size $q$, and suppose a probability distribution is defined on $S$. The goal is to construct non-trivial (eg with $n\neq 1$) sets $S$ with corresponding distributions such that upon independently sampling two elements $s,s'\leftarrow S$, their difference $s-s'$ is non-invertible with probability strictly less than $1/|\mathbb{F}|=1/q$.
For a non-example, let $S=\mathbb{F}_q^{n\times n}$ with the uniform distribution. There are $\prod_{i=0}^{n-1} (q^n-q^i)$ invertible elements in $S$. For any $s\in S$ the number of $s'\in S$ such that $s-s'$ is non-invertible is the same as the number of non-invertible elements of $S$. Therefore, $s-s'$ is non-invertible with probability $$ 1-\prod_{i=0}^{n-1} (q^n-q^i)\Big/q^{n^2} > 1\big/q $$
Any research in this direction? Any ideas for constructing such sets? Any proof such sets don't exist? I'm hoping that requiring differences to be invertible with high probability allows for more options than requiring all differences to be invertible. If you have comments for a similar problem formulated over infinite fields I'd be interested to hear.