Number of points chosen form a polygon to have no isosceles and equilateral triangles.

Question

Let $\Omega$ be a regular polygon with $n$ sides. Let's choose $\Gamma$ a set of vertices, for which any triangle with the vertices in $\Gamma$ is neither isosceles, nor equilateral. Find $\max |\Gamma|$. (Bulgarian Team Selection Test)

Attempt:

I have tried to prove for small cases: $3$, $4$, $5$, $6$, $7$ etc., but no rule confirms, to be proven by induction. And, technically, induction is not possible in this case, because, polygon $\Omega \setminus A$, where $A$ is a vertex, won't be regular.

Answer 1

Upfront warning: this is not really an answer to the question asked, more like a report on my search for sources that couldn't possibly fit in a comment.

Witness the fact that most comments reference integer indices, it is clear that this problem has very little to do with geometry and polygons. The question asked is about avoiding arithmetic progressions in cyclic groups. The triangle aspect of the original question is important though, since it prescribes whether $\{1,6\}$ in $\mathbb{Z}_{10}$ is to be considered an arithmetic progression or not.

Halbeisen (2005) is an example of a treatment that doesn't match our context, but it is still informative, at least for groups with odd cardinalities which do not have this issue. In contrast, Lev (2004) is an example of the opposite choice. At any rate, there seems to be ongoing research on the topic, as can be seen in section 2.2 of this 2011 review by Croot & Lev. I'll quote an interesting excerpt:

(...) it is easy to construct a progression-free set $A \subseteq \mathbb{F}_q$ such that $\vert A \vert = 2^r$: just fix arbitrarily a basis $\{e_1,\dots,e_r\}$ of $\mathbb{F}_q$ over $\mathbb{F}_3$ and let $A := \{\epsilon_1 e_1+\cdots + \epsilon_r e_r: \epsilon_1, \dots,\epsilon_r \in \{0,1\}\}$

Which is a rigorous mathematical formulation of the "greedy algorithm" used in this related math-contest question. Given the fact that the question mentions "Bulgarian Team Selection Test", I'm inclined to think that the expected answer is simply an application of the "greedy algorithm" and would be something along the lines of

$$ 3^{m-1} \leqslant n \leqslant 3^m \implies 2^{m-1} \leqslant \max \vert \Gamma \vert \leqslant 2^m $$

---

Halbeisen, Lorenz; Halbeisen, Stephanie, Avoiding arithmetic progressions in cyclic groups, Elem. Math. 60, No. 3, 114-123 (2005). ZBL1161.11307.

Lev, Vsevolod F., Progression-free sets in finite abelian groups Journal of Number Theory, Volume 104, Issue 1, 162-169 (2004).

Answer 2

The condition that $\Gamma$ be a set of points such that no three of them forms an isosceles or equalateral triangle is the same as saying that the distances between the pairs of points in $\Gamma$ are all distinct. In this case, if $\Gamma$ contains $k$ points then there are $\binom k2 = k(k-1)/2$ distinct distances.

Since $\Omega$ is a regular $n$-gon, it does not contain many distinct distances between its vertices. By symmetry we can count the number of distinct distances starting from a fixed vertex. Observe that there are $\lfloor n/2 \rfloor$ distinct distances.

If we number the vertices $0,1,2, \dots n-1$ going around $\Omega$ in order, we can pick a candidate set for $\Gamma$. Let's choose vertices $0,1,3,6,10,15,\dots, d$ where $d$ is the largest value less than $\lfloor n/2 \rfloor$. It's easy to check that all the distances in $\Gamma$ are distinct. Using the above we can see that if there were any more points in $\Gamma$ there would be a repeated distance in $\Gamma$.

EDIT: As pointed out in the comments this answer is wrong. A similar idea, using the fact that: for each point $p \in \Gamma$, the set of distances from $p$ to the other points of $\Gamma$ must all be distinct, which shows that $|\Gamma| \le \lfloor n/2 \rfloor + 1$.