Let $f : \mathbb R^n \to \mathbb R$ be a differentiable function for which a minimum exists. If $f$ is convex, then
$$\{x \in \mathbb R^n : \nabla f(x) = 0\} = \{x \in \mathbb R^n : f(x) \leq f(y), \; \forall y \in \mathbb R^n\}.$$
However, is the converse statement true? That is, if the above equation holds (and the two sets are non-empty), then is $f$ necessarily convex? Furthermore, would compactness of these sets be relevant?
No, the converse is not correct. Here is a counterexample:
This function is smooth, nonconvex, yet it has a unique global minimiser which satisfies Fermat's condition.
By the way, a convex function does not necessarily satisfy the condition you mentioned - you need additional conditions. Take for example $f(x) = e^x$.
If a (convex or nonconvex) function $f:\mathbb{R}^n\to\mathbb{R}$ is lower semicontinuous and level bounded, then $\inf f$ is finite and its set of minimisers is nonempty and compact. A function $f$ is said to be level bounded if its level sets (the sets $\{x\in\mathbb{R}^n: f(x) \leq a\}$) are bounded for every $a\in \mathbb{R}$ (they might be empty for some $a$).
Update: Another counter-example is the following function
$$ f(x) = \frac{x}{e^{-2x} + 1} $$
Its graph looks a little like the one above.