I wonder if there is any formula of $\frac{d tr(XAX)}{d X}$ when $X$ is symmetric?
I understand that when $X$ is not symmetric, we have $$\frac{d tr(XAX^T)}{dX} = XA^T+XA,$$ and when $X$ is symmetric, we have $$\frac{d tr(AX)}{dX} = A+A^T-A\circ I,$$ where $\circ$ means elementwise product.
However, I am not able to find $\frac{d tr(XAX)}{d X}$ when $X$ is given to be symmetric. Or any hint on the derivative of this formula is appreciated.
For matrix derivatives I find it far easier to think about differentials. If we write $f(X) = \operatorname{tr}(XAX)$ and let $\delta X$ be an arbitrary variation of $X$, then $$df(X)\delta X = \operatorname{tr}(\delta X A X + XA\delta X)$$ from the product rule and linearity of trace. Then we can use the invariance of trace under cyclic permutations to write the above as $$df(X)\delta X = \operatorname{tr}(A X \delta X + X A \delta X)$$ and finally since $\operatorname{tr}(AB) = A^T : B$, where $:$ is the Frobenius product, $$df(X)\delta X = (XA^T + A^TX) : \delta X.$$ Now the notion of a "derivative" of a function with respect to a matrix does not have an entirely standard definition, but if we take it to mean "the matrix whose Frobenius product with $\delta X$ gives you the directional derivative in the $\delta X$ direction" (which I assume is the case from your examples in the OP) then we have our answer: the derivative of $f$ is $$XA^T + A^TX.$$
Important but subtle note: this is the derivative of $\operatorname{tr}(XAX)$ at a symmetric matrix $X$. It is not the same as the derivative of $\operatorname{tr}(X^TAX)$ (even if $X$ is assumed symmetric), although the two functions have the same directional derivative in symmetric directions.