"given only the mean and variance of a distribution, the most conservative assumption that can be made about the distribution is that it is a Gaussian having the given mean and variance"
I've read this in the context of Kalman filters in this paper (at bottom of page 1) and I wonder what this means. What is the most conservative assumption, in what sense?
The Gaussian is the "maximum entropy" probability distribution under those constraints.
It means you are imposing the least other information on the distribution.
Loosely speaking another way to think of it, is that if you put all the distributions in a big bag and then you picked one at a time until you got one that was within some predefined threshold of the mean and variance you want; you'd be much much much more likely to end up with something that looks pretty close to a normal than with anything else.