Intuitive meaning of Limit Supremum?

Question

I am trying to understand the difference between the following two equations:

$$\bar{P} = \limsup_{t \to \infty}\frac{1}{t} \sum_{\tau = 0}^{t-1}E\{P[\tau]\} < \infty$$ and $$\bar{P} = \lim_{t \to \infty}\frac{1}{t} \sum_{\tau = 0}^{t-1}E\{P[\tau]\} < \infty$$

where $\bar{P}$ denotes the average value of P and E stands for expectation. I have previously come across equations like the second one but I am not able to understand when to use equations of the first type. I have read the definition on Wikipedia's Supremum page but I am failing to understand the intuitive meaning of when to use what. The wiki defines it as:

A set A of real numbers (shown as blue balls), a set of upper bounds of A (red balls), and the smallest such upper bound, that is, the supremum of A (shown as a red diamond).

What does a set of upper bounds actually mean? I thought upper bound means the uppermost value but I guess my understanding is flawed. Can someone please tell me the difference between the two and give me some easy to understand example to understand the difference between a normal limit and supremum limit?

Answer 1

I think your question is about the difference between limit, limsup and sup? I will illustrate the differences with an example. Let's consider the sequence: $$3, 4, -7, 2, 1, 7, 1, -1, 1, -1, 1, -1, 1, -1, 1, -1, \dots$$ (The ellipsis indicates that the terms alternate $+1$ and $-1$ forever after the terms I have given explicitly.)

The least upper bound (sup) of this set is $7$ [EDIT: thanks Sivaram]. This is the smallest real number that bounds every term from above. The sequence has no limit. However, the $\limsup$ is $1$.

The formal definition of $\limsup$ is that for each $N$, you consider the "tail" of the sequence, starting at the $N^\text{th}$ position. You take the sup of this tail. Now, take the limit as $N \rightarrow \infty$. This limit exists because the sequences of sup is monotone. (We also allow a limit of $\pm \infty$ so monotonicity suffices.)

Note that if the limit of the $a_n$ exists, then we have $$ \lim a_n = \limsup a_n.$$ This means that in "nice" examples, your two definitions of $\bar P$ are the same. It's only in cases with lots of oscillations that $\lim a_n$ doesn't exist, and you will want to use $\limsup a_n$.

Answer 2

An upper bound is just a number greater than any item of the set. So 2 is an upper bound to (0,1). There are smaller upper bounds, so there are many upper bounds. The supremum is the smallest of these upper bounds, so the supremum of (0,1) would be 1 as all other upper bounds are greater. If we define $f(x)=−x \text{ if } x∈\mathbb{N} \text{ else }0$ there is no limit, but there is a lim sup, which is 0.

Answer 3

As Sam has pointed out, you might need to revisit the definitions of supremum, infimum, limsup, liminf and limit

One of the equivalent way to look at limsup is to look at the set of limit of points of your sequence and find the supremum of the set.

For some reason, students taking analysis for the first time find this definition easier to understand than the other definition, which goes as

$$\limsup_{n \rightarrow \infty} \text{ } x_n = \lim_{n \rightarrow \infty} \sup_{m \geq n} \text{ } x_m$$

For instance for the sequence Sam has written out $$3,4,-7,2,1,7,1,-1,1,-1,1,-1,1,-1,1,-1,\ldots $$ the set of limit points are $\{-1,1\}$. Hence the limsup is $1$ and the liminf is $-1$.

The $\limsup$ and $\liminf$ always exists even if the sequence doesn't have a limit. In fact a nice way to define a limit of a sequence is to define it when both of them are equal to each other and call it the limit of the sequence.

Answer 4

The picture I have in my head of the limit supremum is as follows: if $x_n$ is unbounded, then its limit supremum is $\infty$. Otherwise, graph the points $(n, x_n)$. Place a plank on top of the points which is infinitely long going right, and make sure that the plank is always parallel to the $x$-axis and that it is never allowed to pass through a point. Initially the plank must be at the supremum of the points, but if you move the plank to the right you may move past a supremum, and then the plank can settle down into a lower position. The limit supremum is the limit of the positions that the plank can be at as you keep sliding it to the right.