Intuition on the Johnson-Lindenstauss lemma

155 Views Asked by At

Johnson-Lindenstrauss Lemma states:

Let $N \gg 1$. For any $0 < \varepsilon < 1$ and $m$ points in $\mathbb{R}^N$ and $n > \frac{8 \log m}{\varepsilon^2}$ there exists a linear map $f : \mathbb{R}^N \rightarrow R^n$, such that: $$ (1 - \varepsilon) \Vert u - v \Vert^2 \leqslant \Vert f(u) - f(v) \Vert^2 \leqslant (1 + \varepsilon) \Vert u - v \Vert^2 $$

In other words, there is low-dimensional projection of the data, that almost preserves the distances.

Do I correctly understand, that one can imagine this in a following way : whatever complicated the function from which these points intially came was, whether they lay on some complicated surface, one may choose a hyperplane such that it fits to the data with rather a good precision?

1

There are 1 best solutions below

0
On

I do not see whether your statement can be derived directly from the Johnson-Lindenstrauss-lemma. However in the article

S. Dasgupta, A. Gupta, An elementary proof of the Johnson-Lindenstrauss-lemma, International Computer Science Institute, TR 99-006

it is shown, that $f$ can be chosen to be of the form $\sqrt{\frac{n}{m}}p$, where $p$ is an orthogonal projection to an $n$-dimensional subspace of $\mathbb{R}^N$.

Remark: the set of $m$ points needs not be created by any deterministic process at all, the points can be chosen randomly. Of course since the set is finite this is just a "philosophical" distinction, but thinking about applications of the lemma in data mining it might be of some relevance.