Suppose $\mathbf{X}=[X_1,\ldots,X_K ]^\top$ is a random vector with mean $\theta\mathbf{1}$ and covariance $\mathbf{\Sigma}$. If $\mathbf{w} = [w_1,…,w_K ]^\top$ is a vector which minimizes the objective function $$ \operatorname E\big[ (\mathbf{w}^\top \mathbf{X} - \theta)^2 \big] $$ subject to $$ \mathbf{w}^\top \mathbf{1} = 1 $$ $$ \mathbf{w} \ge \mathbf{0} $$ then it's my understanding that the solution is given by $$ \mathbf{w}_0 = \frac{\mathbf{Σ}^{-1} \mathbf{1}}{\mathbf{1}^\top \mathbf{Σ}^{-1} \mathbf{1}} $$ I'm able to use the Lagrange multiplier method to show that this vector is a critical point. However, I'm not sure how to do a sort of second-derivative check using a similar approach. So, I have two questions:
- Can I extend the Lagrange method to do this second-derivative check?
- Is there a simpler way to show that this vector is in fact a minimum?
You're trying to minimize $\operatorname{var}(\mathbf w^\top \mathbf X) = \mathbf w^\top\Sigma\mathbf w$ subject to a stated constraint on the sum of the scalar entries in $\mathbf w,$ that sum being positive. The existence of at least one local minimum point follows from the fact that it was given that the entries in $\mathbf w$ are all nonnegative and they add up to $1,$ so that we're minimizing a continuous function on a topologically compact space.
The global minimum cannot occur at more than one point for the following reason. Suppose $\mathbf w_1,\,\mathbf w_2$ are two distinct global minimum points. Then $\mathbf w_1^\top \Sigma\mathbf w_1 = \mathbf w_2^\top\Sigma\mathbf w_2$ is the global minimum value. Let $a\mathbf w_1+(1-a)\mathbf w_2$ be a weighted average of the two, so that $0<a<1.$ Then \begin{align} & (a\mathbf w_1+(1-a)\mathbf w_2)^\top\Sigma(a\mathbf w_1+(1-a)\mathbf w_2) \\[10pt] = {} & a^2\mathbf w_1^\top\Sigma\mathbf w_1 + (1-a)^2 \mathbf w_2^\top\Sigma\mathbf w_2 + 2a\mathbf w_1^\top\Sigma\mathbf w_2 \\ & \qquad \text{(Here we used the fact that $\Sigma^\top=\Sigma$.)} \\[10pt] = {} & a^2\mathbf w_1^\top\Sigma\mathbf w_1 + (1-a)^2 \mathbf w_1^\top\Sigma\mathbf w_1 + 2a\mathbf w_1^\top\Sigma\mathbf w_2 \\ & \qquad \text{(since the same minimum value is attained at both points)} \\[10pt] < {} & a^2\mathbf w_1^\top\Sigma\mathbf w_1 + (1-a)^2 \mathbf w_1^\top\Sigma\mathbf w_1 + 2a\mathbf w_1^\top\Sigma\mathbf w_1 \\ & \qquad \text{(Cauchy–Schwarz)} \\[10pt] = {} & \left(\mathbb w_1^\top\Sigma \mathbb w_1\right)\left( a^2 + 2a(1-a) + (1-a)^2 \right) \\[10pt] = {} & \mathbb w_1^\top\Sigma \mathbb w_1. \end{align} This there would be an even smaller value than the smallest value.
(This instance of the Cauchy–Schwarz inequality is strict since the two vectors cannot be scalar multiples of each other.)
If the functions involved are well-behaved enough to show that a local minimum can occur only where Lagrange multipliers exist, then (since a global minimum is a local minimum) you're done.