I do not understand how the James-Stein estimator can perform better (from the point of view of expected Euclidean distance) than the maximum likelihood estimator. I am willing to accept it because I have read the demonstration and I don't have much to say about it, but I have the feeling, when I think about it, that I have reached a contradiction. Can you help me to find the flaw in my reasoning?
To put it simply, let's place ourselves in ℝ³ and consider a Gaussian vector whose variance matrix is the identity; we are trying to estimate the center of the Gaussian vector, and we only have knowledge of one sample.
I have read that the James-Stein estimator consists of "shrinking" the observed point slightly toward (0,0,0).
However, since the choice of the origin is arbitrary, one could as well "shrink" the natural estimate to any other point than the origin.
Let's be even more vicious: it seems to me that, from an algorithmic point of view, if one implements the James-Stein estimator, one can define the shrinkage target only after knowing the sample.
Can I then have fun choosing the shrinkage target so as to obtain the shrinkage coefficient and the shrinkage direction of my choice? In this case, I could manage to shift the natural estimator by, say, 0.42, in the direction of my first axis no matter what. However, it seems to me completely absurd to systematically make this shift which corresponds to nothing instead of using the natural estimator.
Thank you in advance for your clarifications