I want to minimize a function $f(\vec x,\vec y)$, whereby $\vec x$ and $\vec y$ are vectors. If I hold $\vec x$ constant, $f(\vec x,\vec y)$ is convex with respect to $\vec y$, and the reverse is true as well: If I hold $\vec y$ constant, $f(\vec x,\vec y)$ is convex in $\vec x$. However, $f$ is not convex in $\vec x$ and $\vec y$.
The problem does not include any constraints.
For solving the minimization problem, I see three possibilities:
Alternatingly solving the convex problems for $\vec x$ and $\vec y$: Solve the convex problem for $\vec x$ (with $\vec y$ held constant); use the resulting $\vec x$ as the constant in the convex problem for $\vec y$. Solve the convex problem for $\vec y$; use the resulting $\vec y$ as constant in the convex problem for $\vec x$. Proceed until a sufficiently good result is found.
$\vec x_1 = \underset{\vec x}{\arg\min}\{ f(\vec x, \vec y_0) \}$;
$\vec y_1 = \underset{\vec y}{\arg\min}\{ f(\vec x_1, \vec y) \}$;
$\vec x_2 = \underset{\vec x}{\arg\min}\{ f(\vec x, \vec y_1) \}$;
$\vec y_2 = \underset{\vec y}{\arg\min}\{ f(\vec x_2, \vec y) \}$;
$\dots$
Solving the convex problem for $x$ in each step of a general minimization: Set $g(\vec y) := \underset{\vec x}{\min}\{ f(\vec x, \vec y) \}$. Minimize $g$ without using convexity.
- Neglecting the convexity properties: Use an optimization algorithm that does not require convexity.
I see advantages and disadvantages for all three possibilities.
Which of the procedures would converge (in general) the fastest? Why? Do you know of any literature that covers this kind of problem?
My concrete problem:
Doing a maximum likelihood estimate I want to maximize
$$c \left (\ln(\vec p) + \ln(\vec y) - \ln(\vec p + \vec y) \right ) \cdot \vec x - \exp \left( \left(\ln (\vec p) + \ln(\vec y) - \ln(\vec p +\vec y) \right) \cdot \vec x \right )$$
Thereby, $c$ is a real constant, $\vec p$ is a constant vector, and $\vec x$ and $\vec y$ are the vectors I want to find.