Is there a formula for calculating the number of different inscribed triangles in a circle with a given number of equidistante points on it? By "different" I mean that a triangle can't be rotated or reflected onto another.
Consider the following example: If there are $10$ points, the answer is $8$ triangles;
Note: I'v had a look at finding number of triangles inscribed in a circle but it's not exactly what I want. The formula given is for every triangles, regardless of rotation or reflection.
If there are $n$ equally spaced points on the circle, then the three legs of your triangle cross $a$, $b$, and $c$ points, where $a+b+c=n$. Since you're identifying reflections and rotations, permutations of $(a,b,c)$ all represent the same triangle. So you want the number of partitions of $n$ into exactly three parts. This is fairly well known; e.g., see this derivation or this question. The simplest expression seems to be by cases based on $n$ (mod $6$): $$ f(n) = \begin{cases} n^2/12 &\qquad&n\equiv 0 \text{ mod 6} \\ n^2/12 - 1/12 &\qquad& n\equiv 1 \text{ mod 6} \\ n^2/12 - 1/3 &\qquad& n\equiv 2 \text{ mod 6} \\ n^2/12 + 1/4 &\qquad& n\equiv 3 \text{ mod 6} \\ n^2/12 - 1/3 &\qquad& n\equiv 4 \text{ mod 6} \\ n^2/12 - 1/12 &\qquad& n\equiv 5 \text{ mod 6} \\ \end{cases} $$ More briefly, $f(n)$ is the closest integer to $n^2/12$.