Let $F : \mathbb{R}^n \to \mathbb{R}$ be a convex function which is minimized at $x_* \in \mathbb{R}^n$. For a non-empty subset $C \subset \mathbb{R}^n$, I want to solve
$$ x_+ = \arg\min_{x \in C} F(x). $$
My intuition tells me that $x_+$ should be the $x\in C$ such that $\|x-x_*\|_2$ is minimized, i.e., the projection of $x_*$ into $C$.
I couldn't think of a counterexample. For a possible proof, I was thinking of a procedure by contradiction assuming that there exists some $x_0\neq x_+$ in $C$ such that $F(x_0)\leq F(x)$ for all $x\in C$, but I couldn't find the contradiction.
Does anyone know if this is true? Are there any extra conditions for which it is true? I'll appreciate any help.
Here is a contradiction:
Let $F(x) = x$ for $x > 0$ and $F(x) = -1000x$ for $x \leq 0$. Define $C = [-10, -1] \cup [10, 20]$. The point closest to $0$ (the unconstrained minimum) is at $0$, so the closest point in $\ell_2$ norm is $x = -1$. But at that point we have $F(-1) = 1000$ which is worse than $F(10) = 10$.
Even for convex $C$ this is not true. Recall the property of projected points onto convex sets:
$$\langle x - \text{proj}_C(x), y - \text{proj}_C(x)> \leq 0$$
meaning that the direction from a point to it's projection onto $C$ is negatively correlated with every direction entering the set $C$ from the projected point. With this in mind, it's easier to find an example where the projected point is not a constrained minimum. Consider:
$$f(x_1, x_2) = 0.0001*|x_1| + |x_2|$$
and the constraint:
$$C = \{ x | x^T \mathbb 1 \geq 1\}$$
The global minimum is $0$. The projection of the global minimum onto the constraint is $(0.5, 0.5)$ which has $f(0.5, 0.5) = 0.5001$. However, it's clear that there exist feasible points with lower $f$, for example: $(10, 0)$ is feasible and has $f(10, 0) = 0.001$.