what does `ensemble average` mean?

Question

I'm studying this paper and somewhere in the conclusion part is written:

"Since this rotation of the coherency matrix is carried out based on the ensemble average of polarimetric scattering characteristics in a selected imaging window, we obtain the rotation angle as a result of second-order statistics."

Also I've seen the term ensemble average in several other papers of this context.

Now I want to understand the exact mathematical or statistical definition of ensemble averaging not only in this context but the exact meaning and use of ensemble averaging in statistics and mathematics.

I googled the term ensemble average and here in wikipedia we have the definition as

"In statistical mechanics, the ensemble average is defined as the mean of a quantity that is a function of the microstate of a system (the ensemble of possible states), according to the distribution of the system on its microstates in this ensemble."

But I didn't understand this definition because I don't even know what does the microstate of a system or possible states of system mean in mathematics.

Could you please give me a simple definition with some examples for ensemble averaging?
Compare time averaging and ensemble averaging?
And also introduce me some good resources to study more especially resources that can be helpful in image processing too?

Answer 1

The output of a random experiment is generaly treated as a random variable, and we know the definition of the mean (expected value) of a random variable. But in a more general setup, like for example a stochastic process (its just a name, nothing complex about it) the output of a stochastic process is a more general object rather than just a random number. The ensemble is defined as a set of all possible outcomes of a stochastic process, and ensemble average means the expected object (like expected value for random variable) of the stochastic process. Simply speaking it is just the expected value of random variable, but defined for a more general abstract setup.

Answer 2

for analog continous signals, we have time average.Time average is averaged quantity of a single system over a time inetrval directly related to a real experiment. or discrete signals, we have ensemble average. Ensemble average is averaged quantity of a many identical systems at a certain time.

EDIT: A clearer example of this is given on the Wikipedia page on Ergodicity.

"Each resistor has thermal noise associated with it and it depends on the temperature. Take N resistors (N should be very large) and plot the voltage across those resistors for a long period. For each resistor you will have a waveform. Calculate the average value of that waveform. This gives you the time average. You should also note that you have N waveforms as we have N resistors. These N plots are known as an ensemble. Now take a particular instant of time in all those plots and find the average value of the voltage. That gives you the ensemble average for each plot."

Answer 3

I realize this is a late answer to this post, but it still makes the top two to three results on Google for "ensemble average" and an answer has not yet been officially accepted. For posterity, I figured I would try to answer it to the best of my ability in the way that the question has been phrased.

First, it is important to have a broad understanding of what a stochastic process is. It is a fairly simple concept, analogous to a random variable. However, where the value of a random variable can take on certain numbers with various probabilities, the "values" of stochastic processes manifest as certain waveforms (again, with various probabilities). As an example in the discrete world, the outcome of a coin flip could be viewed as a random variable - it can take on two values with roughly equal probability. However, if you recorded the outcome of n coin flips (where n could be any whole number, up to infinity), and were to do so many times, you could view this "set of n coin flips" as a stochastic process. Results where roughly half are heads and half tails would have relatively high probabilities, while results where almost all are heads or almost all tails would have relatively low probabilities. Obviously, there are also continuous random variables and stochastic processes can be either discrete or continuous for both axes (time/trials vs. values/outcomes of each trial).

It is also important to understand expected value. This is even simpler - it's the value that, over a long period of time/many trials, you would expect your random variable to have. It's the mean. The average. Integrate/sum over all time/trials and divide by the amount of time/number of trials.

Now that these two things are covered, the ensemble average of a stochastic process can be explained in simple language and mathematical terms. In the simplest sense, the ensemble average is analogous to expected value. That is, given a large number of trials, it is the "average" waveform that would result from a stochastic process. Note that this means that an ensemble average is a function of the same variable that the stochastic process is. Mathematically, it can be denoted as:

$$ E[X(t)] = \mu_X(t) = \int_{-\infty}^\infty x*p_{X(t)}(x)dx $$

where $p_{X(t)}$ is the PDF of $X(t)$.

You also mentioned the time average for a stochastic process. This is a very different thing, which itself is actually a random variable! The reason for this is that a time average of a stochastic process is simply the average value of a single outcome of a stochastic process. Note that this means that unlike the ensemble average, the time average is not a function, but a value (a number). It can be described mathematically as:

$$ \lim_{T\to\infty} \frac{1}{2T}\int_{-T}^T X(t)dt $$

where $X(t)$ is the stochastic process in question, evaluated at time $t$.

To wrap up: ensemble and time averages are properties of stochastic processes, which are like random variables but take the form of waveforms. Ensemble average is analogous to expected value or mean, in that it represents a sort of "average" for the stochastic process. It is a function of the same variable as the stochastic process, and when evaluated at a particular value denotes the average value that the waveforms will have at that same value. Time average is more like a typical average, in that it is the average value of a single outcome of a stochastic process. It is a random variable itself, as it depends upon which outcome it is being evaluated for (and the outcome itself is random).