Can someone provide an example why/how the Hyperplane Separation Theorems don't hold for non-convex sets?

Question

I understand intuitively that they won't hold, but I can't find any good examples.

Answer 1

For any nonconvex set $S\subset X$ (where $X$ is any vector space, but we can restrict ourselves to $\mathbb{R}^n$ for simplicity), let's find a set $S'$ disjoint from $S$ such that $S$ and $S'$ are not separated by a hyperplane. Take points $x_1, x_2\in S$ such that $x = cx_1+(1-c)x_2\notin S$ for some $c\in (0, 1)$. If $x\in S'$, then $S$ and $S'$ cannot be separated by a hyperplane $P$, as $P$ would partition $\mathbb{R}^n\setminus P$ into two half-spaces, both of which are convex. Therefore, if $x_1$ and $x_2$ are in the same half-space, then $x$ would also be in that half-space, and so $S$ and $S'$ cannot be entirely in opposite half-spaces.

A good example would be a set $S$ in the shape of user363464's Pac-Man, with $S'$ containing a point $x$ inside Pac-Man's mouth. Note that $x$ is surrounded on exact opposite sides by some points in $S$, which therefore cannot both be separated from $x$ by a hyperplane.