In a paper by Jacobson (Nathan Jacobson.Structure of Rings, Volume 37, Part 1. American Math-ematical Soc., revised edition, 1956.), We have a pair of left $R$-modules $A_1$ and $A_2$, and we have $H := \text{hom}(A_1,A_2)$ the linear morphisms from $A_1$ to $A_2$.
Then he defines the following set: Given $S\subset H$, we define
$$S^\perp = \{a\in A_1 \text{ s.t. } aS = \{0\}\}$$
and if $S$ is a subset of $A_1$, we define
$$S^\perp = \{\alpha\in \text{hom}(A_1,A_2)\text{ s.t. } S\alpha= 0\}$$
What does $aS$ mean? is it a notation for $\{s(a)\}$ with $s\in S$?
What does $S\alpha$ mean? is it a notation for $\{\alpha(s)\}$ with $s\in S$?
I don't have a copy of Structure of Rings handy, so I can't verify directly, but there is a possible explanation.
When dealing with noncommutative rings, one must handle sides carefully. If $f$ is a homomorphism of right $R$ modules, it is convenient to use prefix functional notation so that linearity is expressed easily:
$$f(x\cdot r)=f(x)\cdot r$$
Similary with left $R$ modules, it is convenient to use postfix notation so that the linearity expressed this way:
$$(r\cdot x)f=r\cdot(x)f$$
Since Jacobson is handling noncommutative rings and left $R$ modules, I suspect this is what is going on. He is writing his linear maps on the right instead of on the left.
Like what you said (but not exactly), I think it is likely that $S\alpha=\{(s)\alpha\mid s\in S\}$, the image of $S$ via $\alpha$.