Suppose there are two random matrices (distribution unknown), denoted as $A$ and $X$, both in the $\mathbb{R}^{n \times n}$ space. It is known that $\|A\| \leq 1$ (for any $p$-norm) and $E[X]\geq0$. I hope to evaluate the following expectation $$E[A^TXA]$$ to be in terms of $E[X]$. My first instinct is to break it down into element-wise expression but it is quite messy and difficult to reconstruct the matrix form. Is there any smarter way to do this?
Thanks a lot!
Let $X = [x_1,x_2,\cdots, x_n].$ Then $AX = [\sum_{k=1}^{n}a_{1k}x_{k},\sum_{k=1}^{n}a_{2k}x_{k},\cdots, \sum_{k=1}^{n}a_{nk}x_{k}]$ and so $$X^{t}AX= \sum_{i=1}^{n}\sum_{k=1}^{n}a_{ik}x_{i}x_{k}$$ and so $$E[X^tAX] = E\left[\sum_{i=1}^{n}\sum_{k=1}^{n}a_{ik}x_{i}x_{k}\right] = \sum_{i=1}^{n}\sum_{k=1}^{n}E\left[a_{ik}x_{i}x_{k}\right].$$ Since $$E(XYZ) = E(XY)\bar{Z} + Cov(XY, Z)$$ for any three $X,Y,Z$ random variables we have that, $$E[x_{i}x_{k}a_{ik}] = E[x_{i}x_{k}]E[a_{ik}]+Cov(x_ix_k,a_{ik})$$ $$=E[a_{ik}]c_{ki}+Cov(a_{ik},c_{ki}),$$ where $C$ is the covariance matrix of $X.$ So we have that
$$E[X^tAX] =\sum_{i=1}^{n}\sum_{k=1}^{n}E[a_{ik}]c_{ki}+Cov(a_{ik},c_{ki}) $$ $$ = \text{Trace}(E[A]C)+\text{Trace}(K)$$ where $K$ is the covariance matrix of with each entry as $\text{Cov}(a_{ij},c_{mn}).$