Monotonic Function Optimization on Convex Constraint Region

1.1k Views Asked by At

So I have the following function, which I want to maximize: $$f(x_1,...,x_n) = \sum_{i=1}^n\alpha_i\sqrt{x_i}$$ (where all $\alpha_i$ are positive), subjected to the following equality and inequality constraints: $$\sum_{i=1}^nx_i = B$$ $$l_i \leq x_i \leq u_i$$

for $l_i,u_i,B$ all positive.

I want to say that there's a nice geometric way to think about this. In particular, invoking the coordinate transformation $$x_i\mapsto {v_i\over \alpha_i^2} $$, we have $$f(v_1,...,v_n) = \sum_{i=1}^n\sqrt{v_i}$$ and constraints $$\sum_{i=1}^n{v_i\over \alpha_i^2}$$ $$l_i \leq {v_i \over \alpha_i^2} \leq u_i$$

In a global sense, given a fixed sum, $S$, of all the $v_i$ we know from the triangle inequality that the optimal solution is $v_i = {S\over n}$ for all $i$. Is there a way to apply this knowledge of the global solution to finding one within our constraint region (which is the intersection of a hyperplane and a polytope). If there's no easy way to divine a closed form solution, might there be an algorithm suited to this kind of optimization?

Gradient ascent is applicable, but it seems to be too ignorant of what I want to call the nice-ness of the objective function and constraint region.

I don't have much background by way of numerical analysis, so I'd really appreciate any insight.

Edit: I should probably point out that I will eventually be attempting to code this algorithm, so solutions of exponential or worse order are scary to me.

Edit2: I realize this question isn't the most interesting numerical exercise in the world, so I've attached a modest bounty.

1

There are 1 best solutions below

10
On

If we substitute $v_i=w_i^2$ then the constraints define the intersection of a hyperellipsoid (which is an $(n-1)$-dimensional convex surface in $n$-space) with a rectangular block, and the function to optimise is $\sum_iw_i.$ The hyperellipsoid is centered at the origin and its axes are parallel to the coordinate axes.

Since we are only interested in positive $w_i$ the intersection of the $(n-1)$-dimensional hyperellipsoid and the block is a single curved cell bounded by hyperellipsoids of lower dimensions. In order to find the optimum we need to try each dimension in turn, starting with the highest $n-1$ (the interior of the cell).

The direction of the normal to the hyperellipsoid defined by the equation

$$\sum_i\frac{w_i^2}{\alpha_i^2}=B$$

is defined by the coordinates $(w_1/\alpha_1^2,\ldots,w_n/\alpha_n^2).$ The unique point (with positive coordinates) where this coincides with the gradient of the function $\sum_iw_i$ is at the point with coordinates

$$\frac B n(\alpha_1^2,\ldots,\alpha_n^2)$$

If this point lies in the cell or on its boundary (i.e., if its satisfies the constraints defined by the constants $l_i, u_i$) then it defines the maximum of $f$ and we are done.

If this point lies outside the boundary of the cell (i.e., if its violates at least one of the inequalities) then we need to reformulate the problem $2n$ times in one dimension lower: once for every $i=1,\ldots,n$ using the lower bound $\frac{w_i^2}{\alpha_i^2}=l_i$ and again for the upper bound $u_i.$ In this reformulated problem, the block constraint for the $i$-th coordinate is dropped and $w_i$ is otherwise treated as a constant (to be absorbed in the constant $B$ and in the computation of $f$). The constant $\alpha_i^2$ no longer occurs in the solution for the optimum of $f$ on the lower-dimensional hyperellipsoid.

This process is repeated recursively to build a tree of hyperellipsoid segments of ever lower dimension. A node in the tree is a leaf (i.e., has no children) if and only if $f$ reaches a local maximum in that segment.

The global optimum of $f$ is the largest local maximum reached on all the leaves of the tree.

The tree is guaranteed to be finite because $0$-dimensional hyperellipsoids are single points.

The 'worst case' maximum number of nodes in the tree is $2^{n}n!$