Differential of partial minimization

78 Views Asked by At

Let $\mu_1 \in \mathbb{R}^n $ and $\mu_2 \in \mathbb{R}$, $h(\mu_1, \mu_2)$ be a concave function in $\mu_1, \mu_2$, and let \begin{equation} G(\mu_2) = \underset{\mu_1 \geq 0}{max} \,\, h(\mu_1, \mu_2) \end{equation} More specifically, my $h$ function has a special form, which is: \begin{equation} h(\mu_1, \mu_2) = \underset{j= [1, m]}{min} \left(\left(\sum_{i=1}^{n}\mu_1^i (c_i^T v_j) \right) + \mu_2(d^T v_j) \right) \end{equation} where $\mu_1^i$ is the $i^{th}$ coordinate of the vector $\mu_1$. $\lbrace{c_i\rbrace}_{i=1}^{n} \in \mathbb{R}^{p}, d \in \mathbb{R}^p, \lbrace{v_j \rbrace}_{j=1}^{m} \, \in \, \mathbb{R}^p $ are constant vectors. So $h$ is piecewise continuous concave function, since it is a point-wise minimum of family of affine functions.

I am interested in computing $\partial{G(\mu_2)}$ at $\mu_2^*$. I do have with me $\mu_1^* = \underset{\mu_1 \geq 0}{arg \, max } \, h (\mu_1, \mu_2^*)$.

PS: In case you are wondering, this is problem associated with Linear programming duality.

Below is my attempt, which I am not sure is correct. If anyone could help me with the correct solution, that would be great. I also have a few doubts that I mention below.

MY REFINED ATTEMPT: Define: \begin{equation} V(\mu_1, \mu_2) = \underset{v_j\in \lbrace{v_1, \dotsc, v_m \rbrace}}{arg \, min} \, \left(\left(\sum_{i=1}^{n}\mu_1^i (c_i^T v_j) \right) + \mu_2(d^T v_j) \right) \end{equation} where $V(\mu_1, \mu_2)$ is possibly a set, since the minimizer may not be unique. Then we can rewrite $h$ as: \begin{equation} h(\mu_1, \mu_2) = \left(\left(\sum_{i=1}^{n}\mu_1^i (c_i^T v )\right) + \mu_2(d^T v) \right) \end{equation} for any $v \in V(\mu_1, \mu_2)$. Not we can define the sub-differential of $H$ at $\mu_1, \mu_2$ as : \begin{equation} \partial H(\mu_1, \mu_2) = \mathbf{Co}\lbrace{ (c_1^T v, c_2^T v, \dotsc, c_n^T v, d^T v ) | v \in V(\mu_1, \mu_2) \rbrace} \end{equation} where $\mathbf{Co}$ means convex hull. Now since, $G$ corresponds to partial minimization of $h$, from pg 9 of https://web.stanford.edu/class/ee364b/lectures/subgradients_notes.pdf, we can write $\partial{G(\mu_2^*)}$ as: \begin{equation} \label{Eq:1} \partial{G(\mu_2^*)} = \mathbf{Co}\lbrace{ g | (0, g) \in \partial H(\mu_1^*, \mu_2^*) \rbrace} \end{equation} where $g$ is a scalar corresponding to the partial derivative w.r.t $\mu_2$ i.e. the last element in each vector belonging to $\partial H(\mu_1^*, \mu_2^*)$.

Some doubts: So the link provided says that the formula for $\partial G$ is valid if $\mu_2$ is evaluated at an interior point in the domain of $G$. My question, what happens if instead $\mu_2$ is a point on the boundary of the domain of $G$? How do the expressions for subdifferential change?