Sequence of Quadratic Form of Random Variables

43 Views Asked by At

I want to proof that $\mathbb{E}\left[\frac{1}{n}\sigma({u}^{T})w_{n}w_{n}^{T} \sigma(v)\right] \overset{n\rightarrow \infty}{=} \mathbb{E}\left[\sigma({l})\sigma(s)\right]$ with l,s beeing Gaussian and Cov(l,s) depending on Cov(u,v)}.

Proof is done by showing that the expression converges to the mean of the weak distribution limit when $n \rightarrow \infty$ where $u,v,w \in \mathbb{R}^{n}$ and the entries of are i.i.d Gaussian. ws' Covariance Matrix is the Identitiy Matrix. $\sigma$ is applied elementwise. This is a simplication of Lemma 10 of the Appendix of Infinite attention: NNGP and NTK for deep attention networks. For the proof it is enough to show that weak convergence holds using continuous mapping theorem and Slutsky Theorem.

Alternatively more general how show to weak convergence limit of quadratic form of sequence of random variables $\frac{1}{n} {u_{n}}^{T} w_{n} w_{n}^{T} v_{n}$ using Slutsky or whatever works here with w beeing iid gaussian N(0,1) and $u_{n}, v_{n} $ weak converging to $u,v$.