I'm coming to this from a Machine Learning perspective. In ML, we have to compute what we call the loss when our model makes a prediction. This loss gives us a number, allowing us to tell our far or close from the "right answer" (assumed to be provided in some form). Then it can be use to tweak the weights of our model to hopefully make better predictions next time similar inputs are provided.
Therefore one could see each inputs with its associated answer as a probabilistic distribution of events, hence a number of loss functions are probabilistic.
Since these tend to be discrete in nature, a Poisson distribution can be used for regression predictions.
I'm not familiar with Poisson's distribution, but I have extensively used Gaussian. My current grasp is to see Poisson's as essentially a discrete Gaussian. I understand there's a bit of a skew for extreme values (e.g. a true Gaussian has a pretty long tail, which real values may not allow, such as a price of -10$ for instance).
Is that idea far off from reality? If this is far off, any better mental model to get an intuitive grasp of the Poisson distribution?
You are not very far off - in fact, for sufficiently large values of $\lambda$ it is common practice to approximate the Poisson with a normal distribution.