I have been looking at discrete Gaussian distributions and arrived at the following conjecture. I would greatly appreciate a proof (or disproof).
Conjecture. Let $\mu \in [0,1]$ and $\sigma^2 > 0$. Then $$\sum_{n \in \mathbb{Z}} e^{-(n-\mu)^2/2\sigma^2} \le \sum_{n \in \mathbb{Z}} e^{-n^2/2\sigma^2}.$$
Numerical evidence supports this conjecture.
In the continuous analog (i.e., replacing $\sum_{\mathbb{Z}}$ with $\int_{\mathbb{R}}$), this is an equality. Indeed, as $\sigma^2 \to \infty$, the discrete sum becomes a closer approximation of the continuous integral and the inequality almost becomes an equality. (Even for $\sigma^2=1$, the difference is only $2.7 \times 10^{-8}$.)
On the other hand, if we take $\sigma^2 \to 0$, then the infinite sum is dominated by a single term. The inequality becomes, roughly, $$\forall \mu ~~~~~ e^{-\mu^2/2\sigma^2} \le e^0,$$ which is trivially true.
Thus we see that in both extremes ($\sigma^2 \to 0$ and $\sigma^2 \to \infty$) the conjecture holds. This is further evidence that it holds for all values of $\sigma^2$.
These sums can be expressed in terms of Jacobi Theta Functions. However, I don't see how this is helpful.
Just to close the lid on this from my comment, define $f(x)=e^{-x^2/2\sigma^2}$ and $g(x)=f(x-\mu)$. Then Poisson summation implies that \begin{equation} \sum_{n\in \mathbb{Z}} e^{-n^2/2\sigma^2}=\sum_{n\in \mathbb{Z}} \hat{f}(n), \end{equation} while \begin{equation} \sum_{n\in \mathbb{Z}} e^{-(n-\mu)^2/2\sigma^2}=\sum_{n\in \mathbb{Z}} \hat{g}(n)=\sum_{n\in \mathbb{Z}} \hat{f}(n)e^{-2\pi in\mu}. \end{equation}
The last thing to note is the well-known fact that $\hat{f}(\xi)=C_1e^{-C_2\xi^2}$ for some some constants $C_1,C_2>0$ depending on $\sigma^2$, so in particular $\hat{f}(\xi)>0$. The claim then follows by the triangle inequality.