Difference between a random variable and its expected value

90 Views Asked by At

In this paper by Rohe et al.,

Rohe, Karl; Chatterjee, Sourav; Yu, Bin, Spectral clustering and the high-dimensional stochastic blockmodel, Ann. Stat. 39, No. 4, 1878-1915 (2011). ZBL1227.62042.

they show that a certain random matrix has the same eigenvectors as its expected value with high probability. They propose the following illustrative example.

Consider a matrix $W \in \mathbb R^{n\times n}$ with i.i.d. Bernoulli$(1/2)$ entries. We thus have, for $i\neq j$, $$ (WW)_{ij}\sim \mbox{Binomial($n,1/4$)}. $$

Next, they state that

...for any $i,j$, $(WW)_{ij}-\mathbb E[(WW)_{ij}]=o(n^{1/2}\log(n))$. Thus, the Frobenius distance from the squared matrix to its expectation is $$ \|WW/n^2-\mathbb E[WW]/n^2\|_F=\frac{1}{n^2}\sqrt{\sum_{i,j}((WW)_{ij}-\mathbb E[WW_{ij}])^2}=o\left(\frac{\log(n)}{n^{1/2}}\right). $$

However, $WW$ being a random variable, this does not make sense to me. The result of that difference should be yet another random variable. Even if this result is supposed to be a value attained with high probability, I'm not sure I can see how the got that particular asymptotic bound.

What exactly do the authors mean with this notation? How did they get to that asymptotic equality?

1

There are 1 best solutions below

5
On BEST ANSWER

If $WW_{ij} - \mathbb{E}[WW_{ij}] = o(n^{1/2}\log n)$ then $$(WW_{ij} - \mathbb{E}[WW_{ij}])^2 = o(n\log^2 n).$$ Summing this over all $i,j$, we get $$\sum_{ij} (WW_{ij} - \mathbb{E}[WW_{ij}])^2 = o(n^3\log^2 n).$$ Taking the square root, we get $$\sqrt{\sum_{ij} (WW_{ij} - \mathbb{E}[WW_{ij}])^2} = o(n^{3/2}\log n).$$ Dividing by $n^2$, we get $$\frac{1}{n^2}\sqrt{\sum_{ij} (WW_{ij} - \mathbb{E}[WW_{ij}])^2} = o\left(\frac{\log n}{\sqrt{n}}\right).$$

Here, $WW$ is not a random variable, but a particular realization of the random variable. Confusingly perhaps, both are denoted in exactly the same way. Let me give a simpler example. Suppose that $X \sim \mathrm{Bin}(n,1/2)$. Then with high probability, $n/3 \leq X \leq 2n/3$. Therefore with high probability, $|X - \mathbb{E}[X]| \leq n/6$.


Since $WW_{ij} \sim \mathrm{Bin}(n,1/4)$, a Chernoff bound shows that with high probability, $WW_{ij} - \mathbb{E}[WW_{ij}] = o(n^{1/2} \log n)$ for all $i,j$. (In fact, we can replace $\log n$ with any function which is $\omega(\sqrt{\log n})$.) Therefore, the bound in your quote holds whp.