I am not quite sure what Bayesian method want to infer from the data? Frequentists assume the parameters generated the data are fixed constants so they can actually estimate those constants.
Bayesians assume the parameters are random, with their own distribution. If parameters are random, how can we estimate it?? Estimating random quantities probably results in some distribution??? I am not quite understand the philosophy of Bayesisans!!
The way I see it, there are two main differences
1. Bayesian statistics is about belief
In Frequentist statistics, the goal is to find a single model that fits most of the data.
The technical term, is Maximum likelihood estimation.
In Bayesian statistics, the goal is to assign belief to each combination of the parameters in light of the data.
The posterior predictive is in fact, a linear combination of models that are likely.
2. Prior knowledge
Bayesian statistics gives you tools to incorporate your prior knowledge into the model.
Mandatory coin toss example
A coin is tossed 3 times and the results are
H,HandH.A frequentist would conclude that there is a 100% chance of getting a
Hthe fourth time.A smarter frequentist would tell you to get more data and to never bother him again with such small sample.
Where a Bayesian could model her prior knowledge that the coin is likely to be be fair, and get posterior and posterior predictive distributions that tend more in favor of
Hbut do not eliminate the possibility ofT.In the end, it doesn't really matter
Given enough data, the Bayesian prior is worn away, and the frequentist models yield the same results as the Bayesian models.