Anyone can suggest me one or more good books on Statistical Inference (estimators, UMVU estimators, hypothesis testing, UMP test, interval estimators, ANOVA one-way and two-way...) based on rigorous probability/measure theory?
I've checked some classical books on this topic but apparently all start from scratch with an elementary probability theory.
On
Many people still swear by the pair of classics by Lehman et al Theory of Point Estimation and Testing Statistical Hypotheses. If you want something a bit more modern, I like Theory of Statistics by Schervish. It covers both the classical and Bayesian theory, but does not slight either of them. There is also Mathematical Statistics by Shao, that is structured much more like the non measure theoretic textbooks, starting with a whirlwind review of probability theory, and seems to be used as a textbook fairly often judging from the semi-incoherent negative reviews on Amazon.
Probably better than my limited opinion, see the answers to a similar question on MathOverflow.
On
Dienst's recommendations above are all good,but a classic text you need to check out is S.S. Wilks' Mathematical Statistics. A complete theoretical treatment by one of the subject's founding fathers. It's out of print and quite hard to find,but if you're really interested in this subject,it's well worth hunting down.
Be sure you get the hardcover 1963 Wiley edition; there's a preliminary mimeographed Princeton lecture notes from 1944 by the same author and with the same title-it's not the same book,it's much less complete and more elementary. Make sure you get the right one!
On
There's the book by Morris de Groot, and one by Bernard Lindgren. Both have bland titles that I don't remember. I think the former might be "Probability and Statistics" and the latter "Statistical Inference" or something like that.
Lindgren's book contains a proof that the location-scale family of Cauchy distributions admits no coarser sufficient statistic than the order statistic (i.e. an i.i.d. sample sorted into increasing order); maybe that's not a crucial thing but it's something you find frequently asserted but seldom proved, so it stands out in my mind.
Both books cover the topics you've mentioned, although they don't assume you've had measure theory.
Since you mention ANOVA, let me add that if you want to understand the theory, you should know things like the (finite-dimensional) spectral theorem, the singular value decomposition, etc. Many books treat ANOVA and regression without that, so you won't learn why the sampling distributions of test statistics are what they are, etc. I'm not sure which book to recommend for this right now.....
Bickel and Doksum, if you can find the 1977 edition is very good. It is a high-level treatment of statistics from a mathematical (tho not measure-theoretic) standpoint.