Is this expression a quadratic form

Question

I have an matrix expression that basically is of the form:

\begin{equation} tr(B X BX ) \end{equation}

Where $B$ and $X$ and nonsquare matrices. $B$ is $p \times n$, $X$ is $n \times p$.

It seems to me this trace expression is a quadratic form because I got it as part of a longer matrix expression which represents the Hessian term of a Taylor series approximating a matrix function. However, I can't see to get it into the form:

\begin{equation} \text{vec}(X)^T [H] \text{vec}(X) \end{equation}

Where $[H]$ is the square matrix representation of this quadratic form. I want to get it into this form so I can get the eigenvectors of this matrix $[H]$. $\text{vec}(X)$ is the vector of length $pn$ where the columns of $X$ are on top of each other.

Can someone help me confirm whether this trace expression can indeed be converted into an explicit matrix representation?

I know that I can make this into:

\begin{equation} \text{vec}(X^T)^T (B^T \otimes B) \text{vec}(X) \end{equation}

But this is not what I want because it gives me $\text{vec}(X^T)$ on the left! It seems so close yet so far. Thanks.

Answer 1

Just think of the matrix $U = XB.$ Your matrix is $U^2,$ so write down the expression for the trace of the square, then substitute, then find the coefficient of $x_{ij}.$ It's tedious, but not difficult.

Answer 2

You are almost there. Notice that the elements of $\mathrm{vec}(X^T)$ are just a reordering of $\mathrm{vec}(X)$. The two are related by a permutation matrix that is sometimes known as a stride permutation. If $X\in \mathbb{R}^{m\times n}$ then the stride permutation matrix $L_m^{mn}$ satisfies the equation $L_m^{mn}\mathrm{vec}(X)=\mathrm{vec}(X^T)$. Therefore you have $\mathrm{trace}(BXBX)=\mathrm{vec}(X)((L_m^{mn})^T B\otimes B)\mathrm{vec}(X)$. There is also another interesting way to obtain this result that will make future calculations simpler. For matrices $A\in\mathbb{R}^{m_1\times n_1}$ and $B\in\mathbb{R}^{m_2\times n_2}$ define the box product $A\boxtimes B\in\mathbb{R}^{(m_1m_2)\times(n_1n_2)}$ by $$ (A\boxtimes B)_{(i-1)m_2+j,(k-1)n_1+l} = a_{il}b_{jk} "= (A\boxtimes B)_{(ij)(kl)}" $$ then $I_m\boxtimes I_n = L_{m}^{mn}$ and you can also write $$\mathrm{trace}(BXBX)=\mathrm{vec}^\top(X)(B\boxtimes B)\mathrm{vec}(X)$$ The box-product essentially behaves like the Kronecker product and satisfies the following properties: \begin{eqnarray} A\boxtimes(B\boxtimes F) &=& (A\boxtimes B)\boxtimes F \\ (A\boxtimes B)( C\boxtimes D) &=& (A D)\otimes(BC)\\ ( A\boxtimes B)^\top &=& B^\top\boxtimes A^\top\\ (A\boxtimes B)^{-1} &=& B^{-1}\boxtimes A^{-1}\\ \mathrm{trace}(A\boxtimes B) &=& \mathrm{trace}(AB)\\ (A\boxtimes B)\mathrm{vec}(X) &=& \mathrm{vec}(BX^\top A^\top). \end{eqnarray} In addition to these Kronecker and box products can easily be multiplied using the following rules: \begin{eqnarray} (A\boxtimes B)(C\otimes D) &=& (AD)\boxtimes(BC)\\ (A\otimes B)(D\boxtimes C) &=& (AD)\boxtimes(BC)\\ (A\boxtimes B)(C\otimes D) &=& (A\otimes B)(D\boxtimes C)\\ (A\boxtimes B)(C\boxtimes D) &=& (A\otimes B)(D\otimes C) \end{eqnarray} For two two-by-two matrices the box product can explicitly be written $$ A\boxtimes B = \begin{pmatrix} a_{11}b_{11} & a_{12}b_{11} & a_{11}b_{12} & a_{12}b_{12} \\ a_{11}b_{21} & a_{12}b_{21} & a_{11}b_{22} & a_{12}b_{22} \\ a_{21}b_{11} & a_{22}b_{11} & a_{21}b_{12} & a_{22}b_{12} \\ a_{21}b_{21} & a_{22}b_{21} & a_{21}b_{22} & a_{22}b_{22} \\ \end{pmatrix}. $$ and here's an example stride-permutation $$ L_2^6=I_2\boxtimes I_3 = \left( \begin{array}{llllll} 1 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 0 \\ 0 & 1 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 \end{array} \right). $$

Finally, I should mention that there is actually a mistake in the answer. My convention has been to use row-wise concatenation of matrices and the definition above, from which easily follows: \begin{eqnarray*} \mathrm{trace}(BXBX) &=& \mathrm{vec}^\top((BXB)^\top)\mathrm{vec}(X)\\ &=& \mathrm{vec}^\top(B^\top X^\top B^\top)\mathrm{vec}(X)\\ &=& \mathrm{vec}^\top(X) B\boxtimes B^\top \mathrm{vec}(X) \end{eqnarray*}

This answer makes more sense, as the matrix $B\boxtimes B^\top$ is always symmetric, whereas $B\boxtimes B$ is not.

I struggled with questions similar to the one you just made, and the box product has been a great help to me. I hope this was of help to you.