Minimize function / Find α resulting in lowest value for formula

Question

I think it's called minimizing a function, but I'm not sure. What I want is to find the value of $\alpha$ so that the result of the formula (the formula on the right side of the equal symbol) will be the lowest value possible.

\begin{align} V(\alpha) = {} & \alpha^2 x_1 + (1-\alpha)^2 x_2 + 2\alpha(1-\alpha)x_3 \end{align}  

I do already know the values of $x_1, x_2$ and $x_3$. These values are fixed. By using statistics software, I was able to calculate both the value of $\alpha$ and the result of the formula when said $\alpha$ is inserted into the formula. However, it would be nice to know how to solve this by hand/formulas.

I have been trying to search around for a while now, and even though there's many examples to find, I'm stuck.

Could anyone point me in the right direction or show me how it's done step by step?

Thank you.

Answer 1

Notice that $V(\alpha)$ is quadratic in $\alpha$ - you have an $\alpha^{2}$ term, a $(1-\alpha)^{2}=\alpha^{2}-2\alpha+1$ term, and a $\alpha(1-\alpha)=\alpha-\alpha^{2}$ term. So you can complete the square, then set the squared term to zero. However, this will only give you a minimum if the squared term's coefficient is positive.

You can also solve it via calculus, like DMcMor did. In general calculus is the way to go, but for quadratics completing the square is a nice alternative.

Answer in spoilers:

$V(\alpha) = (x_{1}+x_{2}-2x_{3})\alpha^{2} - (2x_{2}-2x_{3})\alpha + x_{2} = a(\alpha - b)^{2} + c$ where $a = x_{1}+x_{2}-2x_{3}$, $b = (x_{2}-x_{3})/a$ and $c = x_{2} - ab^{2} = (x_{1}x_{2} - x_{3}^{2})/a$. Only if $a>0$ will you actually get a minimum, in which case you have $\alpha = b$ and $V(\alpha_{min})=c$.

EDIT:

Since OP has stated $\alpha$ is restricted to the interval $[0,1]$ (correct me if the interval is not closed), it might be the case that the minimum obtained by either this method or calculus will lie outside that interval. In this case, one would pick the value that is as close as possible to that minimum - say you got $1.5$, you would choose $\alpha=1$. If you got $-0.5$, you'd pick $\alpha=0$. You can do this because a quadratic is monotonic either side of its turning point. In fact, if you don't find any turning points in the allowed interval, the function must be monotonic if it is continuous, so the minimum in that interval must be one of the two endpoints (and it's usually simple to determine which).

Answer 2

Take the derivative: \begin{align*} V'(\alpha) &= 2\alpha x_{1} - 2(1-\alpha)x_{2} + 2(1-\alpha)x_{3} -2\alpha x_{3}\\ &=2\alpha x_{1} - 2x_{2} + 2\alpha x_{2} + 2x_{3} - 2\alpha x_{3} - 2\alpha x_{3}\\ &=2\alpha(x_{1}+x_{2}-2x_{3}) -2(x_{2}-x_{3}). \end{align*}

Set $V' = 0$ and solve for $\alpha$:

$$0 = 2\alpha(x_{1}+x_{2}-2x_{3}) -2(x_{2}-x_{3}).$$

This yields

$$\alpha = \frac{x_{2}-x_{3}}{x_{1}+x_{2}-2x_{3}}.$$

At this point you need to verify that this particular $\alpha$ is in fact a local minimum, and as was pointed out in the comments you need to also check the boundary of whatever set $\alpha$ belongs to.

Answer 3

This should be possible as an application of Fermat's theorem, which states that the derivative of a differentiable function is zero at (local) extreme values. Since $x_1$, $x_2$, and $x_3$ are constant, $V$ is a polynomial in $\alpha$, and so we can differentiate in order to obtain \begin{align} V(\alpha) &= \alpha^2 x_1 + (1-\alpha)^2x_2 + 2\alpha(1-\alpha)x_3 \\ &= (x_1 + x_2 -2x_3) \alpha^2 + (-2x_2+2x_3) \alpha + x_2. \\ \implies V'(\alpha) &= 2(x_1+x_2-2x_3) \alpha + 2(x_3 - x_2). \end{align} Setting this equal to zero and solving, we get $$ \alpha = \frac{2(x_2 - x_3)}{2(x_1+x_2-2x_3)} = \frac{x_2 - x_3}{x_1+x_2-2x_3}. $$ Since $V$ is quadratic in $\alpha$, this will be a global minimum if the leading coefficient $x_1+x_2-2x_3 > 0$, and a global maximum if $x_1+x_2-2x_3 < 0$.

In a comment, you specified that you are looking for the minimum on the interval $[0,1]$. If the leading coefficient is positive, and if $\alpha$ ends up in that interval (i.e. if $0 \le \alpha \le 1$), then you are done---the minimum value of $V$ is achieved at $\alpha$. However, if the leading coefficient is negative, or if $\alpha$ is not between 0 and 1, then the minimum value of $V$ will be achieved at one of the endpoints. So, evaluate $V(0)$ and $V(1)$, and take the smallest.

In general, if $f : [a,b] \to \mathbb{R}$ is a differentiable function, the the minimum value of $f$ is found as follows:

(1) Apply Fermat's Theorem: First, solve $f'(x) = 0$. You may get multiple solutions. Each solution gives you a "critical point."

(2) Evaluate $f$ at each of the critical points, and at the endpoints of the interval (i.e. at $a$ and at $b$).

(3) The minimum of $f$ will be the smallest value that you found in step (2).