Confused by certain interpretation of expected value...

Question

I read the following in Stein / Shakarchi's Fourier Analysis book, where they discussed the notion of expectation of a probility density.

"Consider the simpler (idealized) situation where we are given that the particle can be found at only finitely many different points, $x_1, x_2, \ldots , x_N$ on the real axis, with $p_i$ the probability that the particle is at $x_i$, and $p_1 +p_2 + \ldots + p_N =1$. Then, if we knew nothing else, and were forced to make one choice as to the position of the particle, we would naturally take $x = \Sigma x_i p_i$, which is the appropriate weighted average of the possible positions."

This makes no sense to me, for the following reason: Suppose that $x_1 = -1$ and $x_2 = 1$, with $p_1 = p_2 = 1/2$. Then $x = 1/2 - 1/2 = 0$, so their logic dictates that we should pick $0$ as a best guess for where the particle should be. But this makes no sense, because the particular cannot appear at $0$... so I certainly wouldn't pick it.

I mean - x is generically not in the set of possibilities, unless there is one point... help? What do they mean?

Answer 1

The whole purpose of calculating the arithmetic mean of a distribution is to minimize the expectation of the square of the error. It all makes sense if you are penalized one dollar per square of the error. So if you know that tossing 3 fair coins gives you the usual distribution of outcomes, your best guess for the number of heads to show is 1.5, even though that is an impossible outcome. However, it minimizes the expected square of the error. If you change the penalty system, you change the optimum guess. For example, if you know that there is a penalty of \$1000 for guessing a larger number than what happens, and no penalty for guessing a lower number, then you would choose 0. That would minimize your expected penalties. This is also found in real life, where people bias their estimations of things in accordance with the penalties of getting it right or wrong. This is kind of related to game theory.

Answer 2

There are two factors here. First is, as pointed out in the comments by Shalop and Gary, average value of a random variable need not correspond to an actual value assumed by that variable.

There are also issues of interpretation. The well-known joke that says a man whose feet is on fire and head on ice saying he is on the average comfortable.

But I would like to point out another major issue: are the numbers really measurement of any quantities or merely symbols (or tags). The social security number of a person does not indicate any measurable quantity of that person; it is simply a tag. If your positions assignment $x_1=-1$ and $x_2=+1$ are merely tags about two distinct points and about distances, any average will be meaningless (even if it happens to be a value assumed by the random variable).