What will be the expectation of $(X-\mu) ^T A (X-\mu) (X-\mu) ^T$, where $X \in \mathbb{R}^n$, $A \in \mathbb{R}^{n\times n} $ is any symmetric matrix, $E[X]=\mu$, and covariance matrix $E[(X-\mu)(X-\mu)^T]=P$. I know that $E [(X-\mu) ^TA (X-\mu)] =tr(AP) $, and I think $(X-\mu) ^T A (X-\mu) $ will be any scalar quantity (variable).
I got stuck here if someone helps me. I will be forever grateful for him/her.
Let $Y=X-\mu$, it is a random vector with $\mathbb EY=0$. You wish to compute $\mathbb E(Y^TAYY^T)$ (note this is the expectation of a random vector, since $Y^TAY$ is a random scalar), where $A$ is a deterministic symmetric matrix. Writing out everything explicitly, $$ (Y^TAYY^T)_k=\sum_{i,j} Y_iA_{ij}Y_jY_k. $$ Let's compute the expectation term by term. It means we will need to compute $$ A_{ij}\mathbb E(Y_iY_jY_k) $$ for all triples of indices $i,j,k$. There are $3$ possibilities, depending on how many of the indices are equal to each other. Either all three are equal, or all three are different, or two are the same and one is different. (In other words, we are splitting up the cases based on how many elements are in the set $\{i,j,k\}$.)
To proceed, we will need one more assumption that presumably you forgot to state, that the components of the vector $X$ are independent. Or in other words, $Y_i$ and $Y_j$ are independent if $i\not=j$.
Using independence we can see that the expectation is zero in two of the three cases above. In fact, $$ \mathbb E(Y_i^2Y_j)=\mathbb E(Y_i^2)\mathbb EY_j=0,\qquad i\not=j $$ and also $$ \mathbb E(Y_iY_jY_k)=\mathbb EY_i\mathbb EY_j\mathbb EY_k=0,\qquad i\not=j\not=k. $$ Thus, we have shown that $$ \mathbb E(Y^TAYY^T)_k=A_{kk}\mathbb E(Y_k^3)=A_{kk}\mathbb E\bigl((X_k-\mu_k)^3\bigr). $$