Stuck on step in Lagrangian Problem

121 Views Asked by At

For $w,E$ column vectors, $i$ the vector of ones, and $\Sigma$ - an $n\times n$ positive definite symmetric matrix, I am trying to solve the following maximization problem:

$$ \max_{\{ w\}} \left\{ \frac{w^T E}{\sqrt{w^T\Sigma w}}\right\} \quad s.t. \quad w^T i = 1 $$ I form the lagrangian: $$ L = \frac{w^T E}{\sqrt{w^T\Sigma w}}+\lambda(w^T i -1) $$ $$ \frac{dL}{dw}= \frac{E(w^T\Sigma w)^{1/2} - w^TE \Sigma w (w^T\Sigma w)^{-1/2} }{w^T\Sigma w} + \lambda i \overset{\text{set}}{=} 0 $$ but I'm confused how to proceed, usually we put the derivative in terms of $w$ and use the constraint, but that's not so easy in this case, any hints?

update Instead solving the following problem as proposed in the comments: $$ \max_{\{w\}} w^T E \quad s.t. \quad w^Ti=1, w^T\Sigma w=\sigma^2 $$ leads to the result: $$ w = \Sigma^{-1} \frac{E (C\mu -B)+1(A-B\mu)}{AC-B^2} $$ $$ \implies w^T \Sigma w = \frac{C\mu^2-2B\mu + A}{AC-B^2} $$ where: $$ A = E^T \Sigma^{-1}E \quad~~~~~~ B = E^T \Sigma^{-1}1 ~~~~~\quad C= 1^T \Sigma^{-1}1 $$

1

There are 1 best solutions below

0
On BEST ANSWER

Let $\Sigma^{1/2}$ be a square root of $\Sigma$ and perform the variable change $$ u=\Sigma^{1/2}w,\quad a=\Sigma^{-1/2}E,\quad b=\Sigma^{-1/2}i. $$ The problem becomes $$ \max\frac{a^Tu}{\|u\|}\quad\text{subject to } \ b^Tu=1. $$ Since the objective function depends only on the unit vector $\frac{u}{\|u\|}$ we can replace the condition $b^Tu=1$ with $b^Tu>0$ and normalize $u$ to $b^Tu=1$ at the end. We have the following cases

Case 1: $a^Tb>0$ (i.e. $a$ and $b$ pointing at the same half-space). Then the maximum is clearly attained at the same direction as $a$, i.e. $u_0\|a$ and $$ \max=\frac{a^Ta}{\|a\|}=\|a\|. $$ Normalization gives $u_0=\frac{a}{b^Ta}$.

In the original notations: $w_0=\Sigma^{-1/2}u_0=\frac{\Sigma^{-1}E}{i^T\Sigma^{-1}E}$ and $\max=\sqrt{E^T\Sigma^{-1}E}$.

Case 2: $a^Tb\le 0$ (i.e. $a$ and $b$ pointing at the opposite half-spaces). In this case, to maximize the scalar product $a^Tu/\|u\|$ the vector $u$ will again try to point as close to $a$ as possible, but this time coming to the constraint $b^Tu=0$ (no normalization possible, $\|u_0\|=\infty$). Disregarding the constraint $b^Tu=1$, that is replacing it with non-strict $b^Tu\ge 0$, we will get that the best fit will be along the projection of $a$ onto the hyperplane $b^Tu=0$. It means that that is the maximum is not attained, only the supremum $$ \sup=|\text{projection of $a$ onto $b^Tu=0$ along the normal $b$}|= \sqrt{\|a\|^2-\frac{|a^Tb|^2}{\|b\|^2}}. $$