The relationship between AEP and compression

202 Views Asked by At

So i've been reading up on AEP, and trying to get a grasp on it (and to figure out why it is important).

I understand the general definitions, and that the whole idea is the knowlegde of typical sets can be used to determine the average behaviour of a larger sample.

However I don't see the direct link between AEP and data compression. I know there is an implication for compression, but am unable to get it as all explanations seem to high level for my dummies level.

Why is it important for data compression? What does it make possible, sort of?

1

There are 1 best solutions below

2
On BEST ANSWER

It's not important at all for compression - it's important for proving theorems about compression. By establishing that atypical sequences becomes exponentially rare when $n$ gets large, we can focus on constructing (theoretical) compression algorithms that only focus on typical sequences. Since the atypical ones are so rare, we can just map those to some super long codeword and it won't matter since they're so rare anyway and reserve the short codewords to the typical ones. Analysis becomes way simpler this way.

This is clearly not how it's done in real life because it relies on asymptotically long input sequences. It's just a theoretical tool used to establish fundamental limits in compression (and in channel coding too).

Let me know if anything's unclear.