Is there any closed-form expression or atleast an efficient way to calculate this sum?
$$ \sum_{i=1}^{N} (a \cdot i) \bmod{M} $$
we can assume $N$, $a$, and $M$ are large enough such that simple looping is not feasible and that period of the progression is also large.
I am aware of the way to calculate sum of first $N$ modulo natural numbers, but I am not able to extend that idea to this series.
I will try to answer to this question the best i can, first of all we suppose $N,M$ large enough and also that $N > M$ and we also know that $(a\cdot i)\mod{M} = (a) mod M \cdot (i) mod M$ hence : $\sum_{i = 1}^{N} (a\cdot i)\mod{M} = (a)\mod{M}\cdot \sum_{i=1}^{N} (i)\mod{M}$. With that, since $N > M$ because if not then the sum is just the sum from $1$ to $N$ times $(a)\mod{M}$. With that, define $C = \big{\lfloor}\frac{N}{M} \big{\rfloor}$ and $r = N - C\cdot M$. And your sum is equal to :
$$a \mod{M}\cdot\bigg{(} \sum_{i = 1}^{r} i + C\cdot M \bigg{)}$$