Optimization of the function of two variables

Question

I have two functions $f(x,y)$ and $g(x,y)$. I want to minimize the sum of these functions w.r.t $x,y \in (0,1)$. I know that for fixed values of $x$, $f(.,y)$ is a decreasing function while $g(.,y)$ is an increasing function. Hence, I think there must be some optimal point of $y$ with respect to fixed values of $x$. In order to optimize the sum with respect to $x,y$ i follow the following steps.

1- Fix a value of $x_i\in(0,1)$.

2- Optimize the function with respect to $y$ for a fixed $x$ and store the value as $opt(x_i)$ and the optimal value of $y$ as $y(x_i)$.

3- repeat step 1 and 2 until all the values in $(0,1)$ are used.

4- At the end find the minimum among all $opt(x_i)'s$ and use $x_i,y(x_i)$ as optimal values for $x,y$.

I want to know if this strategy is correct?

Answer 1

If you're just looking to find the extreme value of the function, then just as in the one-dimensional case you can differentiate to find the critical points. However, two-dimensional calculus is just a tiny bit trickier than the one-dimensional version.

That said, if for a given $x$, $g_x(y) = f(x, y)$ is decreasing, while for a given $y$, $h_y(x) = f(x, y)$ is increasing, then it's pretty simple to state that you can always get a larger value of the function by sending $x$ towards 0 and $y$ towards 1. It's only if there are places where the function flattens out that you've got a chance of hitting a maximum or minimum value within the region.

Answer 2

My other answer involved missing almost all of the actual question, so let's start afresh.

Let's set $h(x, y) = f(x, y) + g(x, y)$. The first thing to acknowledge is that it is possible for there to be no local extrema of $h$ - for a simple example, let $f(x, y) = -x$ and $g(x, y) = 2x$, then $h(x, y) = x$ has no critical points (although you can state that on the boundary of the region you defined, it does have a supremum).

If the function does have a maximum or minimum point, it will occur when the partial derivatives are both zero (or on the boundary, but since you've defined an open region that's not a valid possibility in this case). So you differentiate with respect to $x$, and find when that derivative is zero (possibly as a function of both $x$ and $y$), and you do the same for $y$, and you work out what point or points those could be, and then you can do a few different things to see whether those points are maxima, minima, or something else.

Your method kind of works, although it will only find an approximate point (since you can only check finitely many $x$ values) and how well you do will depend on how well-behaved the function is outside of the grid of $x$ and $y$ values you are checking.

An alternative approximation method is gradient descent, in which you pick a starting point, work out the gradient of the function at that point, and "walk" a bit in the direction of that gradient until you find yourself circling the same point, which is at least a local critical point if not the actual maximum/minimum.