What's the expectation of inner product between ReLu activation (infinite width)?

88 Views Asked by At

I am wondering how to compute the following two expectation?

  1. $\mathbb{E}_{[u, v]\sim \mathcal{N}(0, \mathbf{K})}[\max(u, 0)\max(v, 0)]$ = ?

  2. $\mathbb{E}_{[u, v]\sim \mathcal{N}(0, \mathbf{K})}[\mathrm{tanh}(u)\mathrm{tanh}(v )]$=?

Please note that both u, and v are scalars. So, the question is the expectation under Gaussian distribution, but with different activation functions. I am for sure that (1) has a closed form solution, while it's unclear for (2).

If anyone can help, I really appreciate it! Thanks!