Relationship between trace norm (a.k.a. Schatten-1 norm) of a matrix and the vector norm of the matrix's row average?

67 Views Asked by At

I'm trying to understand whether a connection exists between two seemingly different optimization problems in machine learning.

Setup: Suppose I have $N$ points $x_1, ..., x_N \in \mathbb{R}^D$, where each point lies on the hypersphere i.e. $||x_n||_2^2 = 1$. I can stack these vectors as rows to form a matrix:

$$X \in \mathbb{R}^{N \times D}$$

1st optimization problem: Maximize the trace norm (also known as the nuclear norm or Schatten-1 norm) of $X$:

$$\arg \max_{x_1, ..., x_N} ||X||_*$$

2nd optimization problem: Minimize the $\ell^2$ norm of the average of the points

$$\arg \min_{x_1, ..., x_N} ||\bar{x}||_2^2 \quad, \quad \bar{x} := \frac{1}{N} \sum_n x_n = \frac{1}{N} \vec{1}_N X^T \in \mathbb{R}^D$$

What relationship exists between these two optimization problems? Is there some way to connect the trace norm to the matrix to the vector norm of the matrix's row average?

Note 1: In the second optimization problem, I'm not necessarily wedded to the $\ell^2$ norm. That problem really just requires the length of the matrix's row average goes to 0, so I could also use $\ell^1$ or maybe another vector norm.

Note 2: If it matters, in my setting, $N$ and $D$ are both large.