Suppose I have a random variable $x \sim p(x)$. I pass $x$ through the rectifier function $y=r(x)$, which returns $x$ if $x>0$ and 0 otherwise.
I am interested in both moments of $y$, i.e. mean an covariance.
I wonder if the following ist true: Lets approximate $p(x)$ by a Gaussian $g(x)$, which has mean and covariance of $p(x)$. Is then: $E_p[y]=E_g[y]$? Same question for covariance of $y$.
Numerical tests seem to support this claim, but I do not see any theoretical justification.
No. Consider the case $X\text{~}U[-\sqrt3,\sqrt3]$ and $g(x)=\frac1{\sqrt{2\pi}}e^{-x^2/2},x\in\mathbb R$. The two distributions have identical mean and variance but$$E_g[r(x)]=\frac1{\sqrt{2\pi}}\int_0^\infty xe^{-x^2/2}dx=\frac1{\sqrt{2\pi}}$$and$$E_p[r(x)]=\int_0^\sqrt3\frac x{2\sqrt3}dx=\frac{\sqrt3}4$$Similarly you can check that the variances come out to be $\frac12\left(1-\frac1\pi\right)$ and $\frac5{16}$ and are unequal.