Simplify sum of quadratic forms to have matrix variable $\sum_i x_i^T A_i x_i$?

1k Views Asked by At

Let $A_i$ be a square symmetric matrix of size $n \times n$ and $x_i$ be the $i$-th row of matrix $X$ of size $m \times n$.

Consider $f(x_1, x_2,...,x_m) = \sum_{i=1}^m x_i^T A_i x_i $.

I need to reformulate $f$ as a function of $X$, which means the formula of $f$ should not contain any $x_i$. Instead, it should contain only $X$.

An attempt is to flatten $X$ as a column vector (called $x$), and create a block diagonal matrix $A$, of which diagonal blocks are $A_i$. Then $f(x_1, x_2,...,x_m) = f(x) = x^T A x$. But I need $X$ to be a matrix having $x_i$ as its rows. So I'm stuck here.

Thank you for your help.

1

There are 1 best solutions below

2
On BEST ANSWER

The $k^{th}$ row of the matrix $X$, expressed as a column vector is $$ x_k = X^Te_k $$ where $\{e_k\}$ are the standard basis vectors for ${\mathbb R}^m$

It will prove convenient to use a colon as an inline product notation for the trace, i.e. $$A:B={\rm Tr}(AB^T)$$ Now perform a step-wise transformation of the function of interest $$\eqalign{ f &= \sum_{k=1}^m x_k^TA_kx_k \cr &= \sum_{k=1}^m e_k^TXA_kX^Te_k \cr &= \sum_{k=1}^m (XA_kX^T):(e_ke_k^T) \cr &= \sum_{k=1}^m (X\otimes X){\rm vec}(A_k):{\rm vec}(e_ke_k^T) \cr &= \sum_{k=1}^m (X\otimes X)\,a_k:e_{(k+km-m)} \cr &= (X\otimes X):\Bigg(\sum_{k=1}^m e_{(k+km-m)}a_k^T\Bigg) \cr &= (X\otimes X):S^T \cr &= {\rm Tr}\Big((X\otimes X)S\Big) \cr\cr }$$ NB: At the beginning of the derivation, the symbol $e_k$ was used as a basis vector for ${\mathbb R}^m$. Midway through the derivation, the symbol $e_{(k+km-m)}$ is used as a basis vector for ${\mathbb R}^{m^2}$.