Following is from Functional Analysis book by Conway :
Convexity of K is 'heavily' used in both proofs of existence and uniqueness of the point such that the infimum is attained. But even analyzing the proof for a long time still I couldn't come up with an example that either existence or uniqueness or both cannot be happened if we drop the convexity from the hypothesis of the statement. Any idea? Thanks!
I believe the question of non-uniqueness has already been settled in the comments. So here is an example of a closed, non-convex set, for which existence fails.
Consider the Hilbert space $\ell ^2$ with its standard orthonormal basis $\{e_n\}_{n\in {\mathbb N}}$, and for each $n$, let $$ y_n=\frac{n+1}ne_n. $$ Defining $$ K=\{y_n: n\in {\mathbb N}\}, $$ observe that for distinct $n$ and $m$, one has that $$ \|y_n-y_m\| = \sqrt{\left(\frac{n+1}n\right)^2 + \left(\frac{m+1}m\right)^2} \geq \sqrt2. $$ We then see that $K$ contains no converging sequence, except for the eventually constant ones, and consequently it is closed. Setting $h = 0$, it is easy to see that $ \text{dist}(h,K)=1, $ but clearly there is no $k_0$ in $K$ realizing that distance.