Decoupling Hadamard Product

Question

Given the matrix-vector product $ A (p \circ q) $, is it possible to decouple $ p $ from the expression, such that $ A (p \circ q) = L p $?.

Ultimately, I need to minimize $ || A (p \circ q) ||^2_2 $ with $p$ being the optimzization parameter. If it helps, $ A = M \otimes I $, where $ I $ is the identity matrix. Any hints/references will be appreciated.

Note: $ \circ $ represents the Hadamard product.

Answer 1

To solve your minimization problem, define the diagonal matrix $Q$ such that $$Q = {\rm Diag}(q),\quad A(q\odot p) = AQp = Lp$$ Write the cost function in terms of this new matrix. Then find its differential and gradient. $$\eqalign{ \phi &= Lp:Lp \cr d\phi &= 2Lp:L\,dp \cr &= 2L^TLp:dp \cr \frac{\partial\phi}{\partial p} &= 2L^TLp \cr }$$ Setting the gradient to zero yields $$\eqalign{ L^TLp &= 0 \quad\implies\quad p &= \big(I-L^+L\big)w \cr }$$ where $L^+$ denotes the pseudoinverse of $L$ and $w$ is an arbitrary vector.

NB: In some of the steps above, a colon is used to denote the dot product.

Answer 2

If $p = (p_1,\dots,p_n)$, and $q = (q_1,\dots,q_n)$, then $p \circ q = (p_1q_1, \dots, p_nq_n)$. The $i^\mathrm{th}$ component of $A(p \circ q)$ is then given by

$$ \sum_j a_{ij} p_jq_j $$

If you define your matrix $L$ to have entries $L_{ij} = a_{ij} q_j$, then your desired identity will hold: $A(p \circ q) = L p$. It looks like you take the hadamard product of each row of $A$ with $q$ to get the rows of $L$.