I am working on an optimization problem, and I am a little bit cofused. I know how to solve the constrained problems – using Lagrangian Multiplier – but what about unconstrained ones? (I know how to solve the problem in Matlab using a derivative-free method.) E.g.,
$$ f(\bar w) = f(x,y,z) = x^2+xy+y^2+yz+z^2-6x-7y-8z+9 $$
On
You really do have to check if $$ F(x,y,z) = 2x^{2}+xy+y^{2}+yz+z^{2}-6x-7y-8z-9 $$ has a max, a min, or neither. There's no simple derivative test for 3 variables. There's a trick that can be used to get rid of the cross terms. First, let $x=x'+\alpha y$, and choose $\alpha$ so that the $x'y$ term vanishes: $$ F(x'+\alpha y,y,z)=(2x'^{2}+4\alpha x'y+2\alpha^{2}y^{2})+(x'y+\alpha y^{2})+y^{2}+yz+z^{2} $$ $$ -6x'-(7+6\alpha)y-8z-9. $$ The coefficient of $x'y$ is $4\alpha+1$. Letting $\alpha=-\frac{1}{4}$ does the job and gives $$ F(x'+\alpha y,y,z)=2x'^{2}+\frac{7}{8}y^{2}+yz+z^{2}-6x'-\frac{11}{2}y-8z-9. $$ Similarly, let $z=z'+\beta y$ with $\beta=-1/2$ to obtain $$ F(x'+\alpha y,y,z'+\beta y)=2x'^{2}+\frac{5}{8}y^{2}+z'^{2}-6x'-\frac{3}{2}y-8z'-9. $$ All of the transformations are invertible. So, now you can see that the problem has a minimum, and the minimum is obtained by setting the partial differential equations in $x',y,z'$ to 0. The resulting equations for $x',y,z'$ are trivial: $$ 4x'-6 = 0, \;\;\; \frac{5}{4}y -\frac{3}{2}=0,\;\;\; 2z'-8=0. $$ Finally, you obtain $(x,y,z)$ from $x=x'-\frac{1}{4}y$, $y$, $z=z'-\frac{1}{2}y$.
WARNING: Check my algebra. The ideas work, but ... .
There's no need to apply method of Lagrangian multiplier. But you can still apply it by introducing a dummy variable, say $t=0$. Then you can write $$\mathcal L(w;t)=f(w)-\lambda t$$