Find analytically, $s \in \mathbb R^m$ which maximizes $s^Tg$ subject to $\|s\|_0 \le k$ and $a_j \le s_j \le b_j$ for all $j$.

47 Views Asked by At

Let $a_1,b_1,\ldots,a_m,b_m \in \mathbb R$ such that $a_j \le 0 < b_j$ for all $j$. Let $k \in \{0,1,\ldots,m\}$. Let $g \in \mathbb R^m$ be fixed. Given $s \in \mathbb R^m$, let $\|s\|_0$ denote the number of nonzero coordinates of $s$.

Question. Find analytically, the solution to $$ \max_{s \in \mathbb R^m}s^Tg\text{ subject to }\|s\|_0 \le k \text{ and } a_j \le s_j \le b_j\;\forall j. \tag{*} $$


Solution for the symmetric case where $a_j = -b_j$ for all $j$.

With the change of variables $u_j \leftarrow s_j / b_j$ and $v_j \leftarrow b_jg_j$ for all $j$, our problem is equivalent to

$$ \max_{\|u\|_0 \le k,\;\|u\|_\infty \le 1} u^Tv = \max_{J \subseteq {m\choose k}}\max_{\text{supp}(u) = J,\;\|u\|_\infty \le 1}u^Tv = \max_{J \subseteq {m\choose k}}\|v_J\|_1, \tag{1} $$

which is solved by taking $J$ to be the indices of the top-$k$ largest-in-absolute-value entries of $v$. Here, $v_J$ is the restriction of the vector $v$ on the subset indices in $J$.

Let $T_k(v)$ is the hard-thresholding of $v$ at level $k$, i.e the $m$-vector obtained from $v$ by only keeping the entries with the top-$k$ absolute values. More precisely, if $\sigma \in \mathfrak S_m$ is a permutation which sorts the absolute values of the coordinates of $v$ in descending order, then

$$ T_k(v)_j = \begin{cases}v_j,&\mbox{ if }\sigma(j) \le k,\\0,&\mbox{ else.}\end{cases} $$

Then, problem (1) is solved by taking $u = u^*$, where

$$ u^* = \arg\max_{\|u\|_\infty \le 1}u^TT_k(v) = \text{sign}(T_k(v)). $$

Here, the sign is computed coordinate-wise and

Therefore, our original problem (*) is solved by taking $s_j=b_j\text{sign}(T_k(v)_j)$ for all $j$.

Computational note. All in all, in this symmetric case, the analytic solution can be computed in $\mathcal O(m)$ time and space.


Edit: Solution to general problem (inspired by user Angela Richardson's answer)

Define the vectors $c, v \in \mathbb R^m$ by setting

$$ c_j = \begin{cases}b_j,&\mbox{ if }g_j \ge 0,\\a_j,&\mbox{ else,}\end{cases} $$

and $v_j := c_jg_j$ for every $j$. Then, our problem can be rewritten as $\max_{J \subseteq {m\choose k}}c_J^Tg_J$.

Thus, if $T_{v,k}(c)$ is the vector obtained from $c$ replacing every coordinate by zero except the coordinates corresponding to the $k$ largest coordinates of the vector $v$, then our original problem (*) is solved by taking $s=T_{v,k}(c)$.

Computational note. Just as in the symmetric case, all computations here can be done in $\mathcal O(m)$ time and space.

1

There are 1 best solutions below

2
On BEST ANSWER

The solution in the asymmetric case is very similar the symmetric case.

One of the $s$ which maximises $s^Tg$ will satisfy $s_j\in\{0,a_j,b_j\}$ for all $j$.

So combine the values of $a_jg_j$ and $b_jg_j$ into one big list. If the $k$ largest values in this list are nonnegative, then the maximum value of $s^Tg$ is the sum of these $k$ largest values.

If the $k$th largest element in the list is negative, then find the largest $l$ such that the $l$ largest elements are all positive and the maximum value of $s^Tg$ is the sum of these $l$ largest elements.

For example, if $k=3$ and the $k$ largest values are $a_5g_5,b_7g_7,b_1g_1$ respectively and $b_1g_1\ge 0$ then $s_1=b_1,s_5=a_5,s_7=b_7$ and all the other coordinates of $s$ are zero. If $b_1g_1<0$ but $b_7g_7\ge 0$ then $s_5=a_5,s_7=b_7$ and all other coordinates are zero.