I am reading the book about statistics at the moment and they compare Gaussian distribution with Student's. And one of the items is that Gaussian distribution is more sensitive to outliers.
I am not sure I 100% understand with what it means. If anyone can take a look at my own explanation of this it would be very appreciated.
Let's say that we have some dataset. Then we are trying to find a "model", underneath this data. For doing that we are "learning" parameters of a distributions. After we got a parameters, we introduce some outliers, and applying the same learning procedure again. And after that comparing parameters of this two models (before introducing outliers and after).
So they mean that the difference between this two models will be bigger for Gaussian.
But what confuses me is that the learning process may be organised in a different ways. And I think that the one that will give us a shift towards outliers should be based on the likelihood maximisation idea. Only in this case (again, I'm not sure here) we would say on some iteration of our learning algorithms "hey, with current parameters, this outlier has such a tiny probability, that we need to shift our mean to maximise likelihood".
Is this explanation correct?
Thank you!