Calculate Hadamard Product of a Symmetric Matrix on Itself

Question

Given a positive definite symmetric matrix $K$, what is its Hadamard product ?

Here's what I did:

\begin{align*} (K \circ K)_{ij} &= K_{ij} K_{ij} \qquad \text{(Hadamard product definition)}\\ &= (U \Sigma U^T)_{ij} (U \Sigma U^T)_{ij} \qquad \text{(Real symmetric matrices Eigendecomposition on K)} \\ &= (\sum_{k} \sum_l U_{ik} \Sigma_{kl} U_{jl})(\sum_{k} \sum_l U_{ik} \Sigma_{kl} U_{jl})\\ &= (\sum_{k} \sum_l U_{ik}^2 \Sigma_{kl}^2 U_{jl}^2)\\ &= (U^2 \Sigma^2 U^2)_{ij} \end{align*}

where $U$ is a orthogonal matrix and $\Sigma$ is a diagonal matrix.

Is it right?

(Edited) Addition:

If it is not right, could anyone shed lights of solutions to this problem? The purpose of performing such transformations are to prove $K \circ K$ is positive definite.

Answer 1

I don't know of any results for a mixed matrix/hadamard product of general matrices, but there is a rule for a product of rank-1 matrices, e.g. $$(ab^T)\circ(xy^T)=(a\circ x)(b\circ y)^T$$ We also know that the Hadamard product distributes over addition $$(A+B)\circ(X+Y)=A\circ X+A\circ Y+B\circ X+B\circ Y$$ This suggests the following approach: write the SVD as a sum of rank-1 terms and multiply $$\eqalign{ USV^T &= \sum_{i=1}^rs_iu_iv_i^T \cr K\circ K &= \Big(\sum_{i=1}^rs_iu_iv_i^T\Big)\circ\Big(\sum_{i=j}^rs_ju_jv_j^T\Big) \cr &= s_1s_1(u_1\circ u_1)(v_1\circ v_1)^T + s_1s_2(u_1\circ u_2)(v_1\circ v_2)^T + \ldots + s_1s_r(u_1\circ u_r)(v_1\circ v_r)^T \cr &+\,\, s_2s_1(u_2\circ u_1)(v_2\circ v_1)^T + s_2s_2(u_2\circ u_2)(v_2\circ v_2)^T + \ldots + s_2s_r(u_2\circ u_r)(v_2\circ v_r)^T \cr &+\,\,\ldots \cr &+\,\, s_rs_1(u_r\circ u_1)(v_r\circ v_1)^T + s_rs_2(u_r\circ u_2)(v_r\circ v_2)^T + \ldots + s_rs_r(u_r\circ u_r)(v_r\circ v_r)^T \cr\cr }$$ Since the Hadamard product is commutative, this can be slightly simplified to collect the cross-terms, e.g. $$s_1s_2(u_1\circ u_2)(v_1\circ v_2)^T+s_2s_1(u_2\circ u_1)(v_2\circ v_1)^T=2s_1s_2(u_1\circ u_2)(v_1\circ v_2)^T$$ but that's about it, and probably illustrates the reason that there's no simple expression for general matrices.

Answer 2

In general

$$ \left(\sum_{k} \sum_l U_{ik} \Sigma_{kl} U_{jl}\right)\left(\sum_{k} \sum_l U_{ik} \Sigma_{kl} U_{jl}\right) \ne \sum_{k} \sum_l U_{ik}^2 \Sigma_{kl}^2 U_{jl}^2 $$

But rather

$$ \left(\sum_{k} \sum_l U_{ik} \Sigma_{kl} U_{jl}\right)\left(\sum_{k} \sum_l U_{ik} \Sigma_{kl} U_{jl}\right) = \sum_a \sum_b \sum_k \sum_l U_{ia} \Sigma_{ab} U_{jb}U_{ik} \Sigma_{kl} U_{jl} $$

I believe

$$ \Sigma = \begin{bmatrix} 2 & 0 \\ 0 & 1 \end{bmatrix}, U = \begin{bmatrix} \cos(\pi/6) & \sin(\pi/6) \\ \sin(-\pi/6) & \cos(\pi/6) \end{bmatrix} $$

Serves as a counterexample.