Find orthonormal $\{b_i\}$ such that $\mathcal{A} = \sum_i \sum_j \lambda_i \mu_j b_i b_j^T$

47 Views Asked by At

Suppose $\{b_i\}_{i = 1}^d \subset \mathbb{R}^n$ is an orthonormal set of vectors, $d \leq n$, and assume that a matrix $\mathcal{A} \in \mathbb{R}^{n \times n}$ has the following form:

$$\mathcal{A} = \sum_{i = 1}^d \sum_{j = 1}^d \lambda_i \mu_j \; b_i b_j^T = \sum_{i = 1}^d \sum_{j = 1}^d \lambda_i \mu_j \; b_i \otimes b_j$$

where $\lambda_i, \mu_j \in \mathbb{R}$.

Then the problem I am interested in is how one can recover the vectors $b_i$ (at least some set of orthonormal vectors, since such a decomposition might not be unique).

My question is whether this is a problem people have already looked at before. If not, here is an idea that I have:

ATTEMPT:

Notice that if $\{b_i\}$ is an orthonormal set, then any matrix of the form $b_i b_j^T$ will either have its columns or rows summing to $0$, meaning $\mathbf{1}_n^T b_i b_j^T \mathbf{1}_n = 0$, where $\mathbf{1}_n \in \mathbb{R}^n$ is a vector of ones.

If we find a unit vector $x \in \mathbb{R}^n$ where $x = b_k$ for some $k$, then:

$$\mathcal{A}xx^T = \mathcal{A}b_k b_k^T = \sum_{i = 1}^d \sum_{j = 1}^d \lambda_i \mu_j \; (b_i b_j^T)(b_k b_k^T) = \sum_{i = 1}^d \lambda_i \mu_k \; b_i b_k^T$$

which is a sum of matrices of the form mentioned above. Thus, a minmizer to the problem:

$$\text{minimize} \; \; \mathbf{1}_n^T \mathcal{A} x x^T \mathbf{1}_n$$ $$\text{subject to} \; \; \|x\| = 1$$

exists and is equal to $0$.

By solving this problem, we can find at least one of the $b_i$'s and maybe find the others somehow.

The potential problem that I see is that this problem might not be convex. And this is as far as I have gone so far.

1

There are 1 best solutions below

2
On BEST ANSWER

If $\mathcal A$ is non-zero, your sum can be written as $$ \mathcal A = \sum_{i = 1}^d \sum_{j = 1}^d \lambda_i \mu_j \; b_i b_j^T \\= \pmatrix{b_1 &b_2 & \cdots & b_d} \pmatrix{\lambda_1 \mu_1 & \lambda_1 \mu_2 & \cdots & \lambda_1 \mu_d\\ \lambda_2 \mu_1 & \lambda_2\mu_2 & \cdots & \lambda_2 \mu_d\\ \vdots & \vdots & \ddots & \vdots \\ \lambda_d \mu_1 & \lambda_d\mu_2 & \cdots & \lambda_d \mu_d}\pmatrix{b_1 &b_2 & \cdots & b_d}^T\\ = B \lambda \mu^T B^T = (B\lambda)(B\mu)^T $$ Here, $B$ is the matrix with columns $b_i$, $\lambda,\mu$ are the column-vectors with entries $\lambda_i,\mu_i$, where we define $\mu_i = \lambda_i = 0$ for $i > d$.

It follows that $\mathcal A$ is necessarily a rank-1 matrix.

Suppose that the $(i,j)$ entry $a_{ij}$ of $\mathcal A$ are non-zero. One choice of $B,\lambda,\mu$ that works is as follows: take $b_1,b_2$ to be orthonormal vectors whose span contains $\operatorname{col}(\mathcal A) + \operatorname{col}(\mathcal A^T)$. Extend to an orthonormal basis. Select $\lambda_1,\lambda_2$ so that $\lambda_1 b_1 + \lambda_2 b_2$ is the $j$th column of $\mathcal A$, and select $\mu_1,\mu_2$ so that $\mu_1 b_1 + \mu_2 b_2$ is the $i$th row of $\frac 1{a_{ij}}\mathcal A$. Extend the columns of $B$ to form an orthonormal set, and set the remaining entries of $\mu,\lambda$ to be zero.


The choices of $\mu,\lambda,B$ here are highly non-unique. For any $d \times d$ orthogonal matrix $U$, we find that $$ \mathcal A = (BU)(U^T\lambda)(\mu^T U)(BU)^T $$ is an alternative representation.