Let assume $\theta,\theta'\in R^d$ (are two fixed $d$ dimensional real-valued vectors) and $x\sim N(0,I_d)$ (is a d-dimensional random vector comes from standard normal distribution). Now, I am wondering about the quantity
$$E_{x}[\sigma(\theta^Tx)\sigma(\theta'^Tx)],$$ where $\sigma$ is the ReLU function defined as $\sigma(z):=\max (0,z)$ for real-valued $z$. Basically I am interested to know the expected value of the product of two ReLU functions. Any help would be appreciated. I am also wondering if maybe some conditions are needed for $\theta,\theta'$ for the integral to make sense?