My goal is to minimise:
$$\min_w\left( w'(a \cdot\mathrm{diag}(1/|w_1|,...,1/|w_n|+ X)w+b'w+x'w\right)$$
where $X\in\mathbb{R}^{n\times n}$ semi-positive definite, $b,w,x\in\mathbb{R}^n$ and $a>0$. Also, $w_1,...,w_n\neq 0$. I am having trouble with the following expression:
$$\frac{\partial}{\partial w}\left(w'(a\cdot \mathrm{diag}(1/|w_1|,...,1/|w_n|+ X)w\right)$$
I am aware that $w'Aw=2A$, if $A=A'$. In the above case $A$ is a function of $w$, so I am not sure how to go about this. Could anyone explain to me please, how to deal with this scenario?
Let's use a colon to denote the trace/Frobenius product, i.e. $$A:B = {\rm Tr}(A^TB)$$ and define the vectors $$\eqalign{ c &= b + x,\quad s &= {\rm sign}(w) \cr }$$ and the matrices $$\eqalign{ S &= {\rm Diag}(s),\quad W &= {\rm Diag}(w) \cr }$$ as well as the scalar $\alpha = a$
The following relationships will prove useful. $$\eqalign{ SW &= WS = {\rm abs}(W) \cr W1 &= w \implies W^{-1}w=1 \cr S1 &= s \implies S^{-1}s=1 \cr A\!:\!B &= B\!:\!A = A^T\!:\!B^T \cr A\!:\!BC &= AC^T\!:\!B = B^TA\!:\!C \cr \cr }$$ Write the cost function in terms of these new variables. $$\eqalign{ \phi &= \alpha SW^{-1}:ww^T + X:ww^T + c:w \cr &= \alpha SW^{-1}w:w + X:ww^T + c:w \cr &= \alpha S1:w + X:ww^T + c:w \cr &= (\alpha s+c):w + X:ww^T \cr }$$ Then find its differential and gradient. $$\eqalign{ d\phi &= (\alpha s+c):dw + X:(dw\,w^T+w\,dw^T) \cr &= (\alpha s + c + Xw + X^Tw):dw \cr \frac{\partial\phi}{\partial w} &= \alpha s + c + Xw + X^Tw \cr }$$