Consider the following problem with a symmetric matrix $\Sigma \in \mathbb R^{n\times n}$ and vector $x \in \mathbb R^{n\times 1}$ and the function $f: \mathbb R^{n\times 1} \to \mathbb R$:
$$f(x) = (x\otimes x)'\left((\Sigma x\big)\otimes(\Sigma x)\right),$$
where $\otimes$ denotes the element-wise product. I want to compute the derivative with respect to $x$, which I get to be:
$$\frac{\partial f(x)}{\partial x}=\frac{\partial (x\otimes x)'}{\partial x}\left((\Sigma x)\otimes(\Sigma x)\right)+(x\otimes x)'\frac{\partial \left((\Sigma x)\otimes(\Sigma x)\right)}{\partial x}.$$
I able to calculate the above vector of derivatives, but I am wondering if it is not possible to simplify the expression?
Define a new vector variable $$ y = (Ax)\circ x$$ Note that I've used a more conventional notation for the elementwise/Hadamard product. I'll also be using a colon to denote the trace/Frobenius product, i.e. $${\rm tr}(A^TB) = A:B$$
First write the function in terms of this new variable and then find the differential and gradient $$\eqalign{ f &= y:y \cr df &= 2y:dy \cr &= 2y:\big(x\circ(A\,dx) + (Ax)\circ dx\big) \cr &= 2(y\circ x):(A\,dx) + 2(y\circ(Ax)):dx \cr &= 2\Big(A(x\circ y) + (Ax)\circ y\Big):dx \cr \frac{\partial f}{\partial x} &= 2\Big(A(x\circ y) + (Ax)\circ y\Big) \cr\cr }$$ It is helpful to know that the Hadamard and Frobenius products commute with themselves and each other $$\eqalign{ A:B &= B:A \cr A\circ B &= B\circ A \cr A:(B\circ C) &= (A\circ B):C \cr }$$