i have the following two general convex optimization problems: $$\min_X \sum_i f_i(x_i,D)$$ $$\min_{D.W} \sum_i g_i(x_i,W,D)$$ $X,D,W$ are parameter matrices and $Y$ is the data samples matrix. $f,g$ are convex functions and deferentiable respect to the above matrices.
As you see these problems are coupled and the solution to one can effect the other. For example finding optimum $X$ for the first problem provides different set of $g_i$ functions (for instance imagine $g_i=x_iWD$).
I like to find the optimum point $X^*,D^*,W^*$ for the whole problem. However the 2nd optimization is the main goal of the problem, so maybe we can call it a bilevel optimization, but not sure about it!
I tried a solution as follow:
1- Finding best $X^*$ matrix based on the first problem.
2- Finding best $W^*$ matrix based on the 2nd problem.
3- Finding best $D^*$ matrix based on the 3rd problem.
and repeating 1-3 until convergence. The problem is that the result is sub-optimal as i tried it with toy-data and it failed to find the better optimum point that exist.
also i'm not sure if the whole strategy is properly chosen to solve the above problem.