Groups are rarely mentioned during the first semester, but they appear in linear algebra exams. This is an example and my idea which undoubtedly lacks formal language:
If possible, find numbers $a,b \in \mathbb R, z\in \mathbb C$ so that the set $$S=\{a-bi,a+bi,z\},$$ consisting of different complex numbers forms a multiplicative group. Justify the existence or non-existence of such numbers. This is my attempt, please, correct me. As a student, I have to be $mathematically$ $literate$.
Closure under multiplication, the existence of identity and inverse element imply one of the three elements is 1 an the other two are inverse.
$$a\pm bi\notin \mathbb R$$
because there would be two equal elements, therefore, $z=1$ and
$$a\pm bi=\frac{1}{a\mp bi}$$
Since the set is closed under multiplication, any power of each element must be in that set. Let me take an arbitrary number x and its inverse:
$x^z\in S, z\in \mathbb Z \implies x^z=\frac{1}{x} \implies x^{z+1}=1.$
If I write x in the trigonometric form:
$$x=\lvert x\rvert(\cos\alpha + i\sin\alpha)$$
I have to find an odd exponent so that
$$x^z=1, x\ne\pm1.$$
$\implies$ for example:
$$x^z=\lvert x\rvert(\cos3\alpha+i\sin3\alpha)=1$$
For $\alpha=-\frac{\pi}{3}$:
Is the solution:
$$S=\{\frac{1}{2}+\frac{\sqrt3}{2}i,\frac{1}{2}-\frac{\sqrt3}{2}i,1\}$$
Every critic is helpful.
Groups are mentioned in linear algebra courses, as any vector space is an abelian group, and also invertible matrices form so called global linear group. However, your question has little to do with linear algebra, and I sincerely doubt it would appear on any linear algebra exam. It is more appropriate for introductory course in group theory/algebra.
That said, let's take a look at what you wrote.
You correctly conclude that if such a group exists, then $z = 1$ and that $\omega = a + bi$ and $\overline\omega = a - bi$ must be inverses to each other.
Again, you are correct that any integer power of a group element must be contained in the group, but I can't really follow your line of thought here. You are looking at solutions of $x^n = 1$ (it is customary to use $n$ for integers and $z$ for complex numbers) which is certainly correct idea, but then out of nowhere, you set $n = 3$. Again, this is correct, but you don't explain anywhere why this is significant.
An easy way to justify this is with little bit of group theory. Since your group has $3$ elements, for any group element $x$ we must have $x^3 = 1$. This equation has precisely $3$ distinct solutions in $\mathbb C$, so if your group exists, its elements are third roots of unity. And these do form a subgroup of $\mathbb C$ (which you didn't prove), since they are closed under multiplication and inverses: $$(\alpha\beta)^3 = \alpha^3\beta^3 = 1\cdot 1 = 1,\ (\alpha^{-1})^3 = (\alpha^3)^{-1} = 1^{-1} = 1.$$
We can also arrive at the same conclusion without this general fact from group theory.
Note that $\omega^2\in\{1,\omega,\overline\omega\}$. Consider cases:
And finally, what you wrote aren't solutions to $x^3 = 1$. Note that $(\frac{1}{2}+\frac{\sqrt3}{2}i)^3 = -1$. What you want is not the angle $\alpha = -\pi/3$, since if you triple it you get $3\alpha = -\pi$, and the argument of $1$ is $0$, not $-\pi$. The trick is to write $1 = \cos 2\pi + i\sin 2\pi$, so you want $\alpha = 2\pi/3.$ With that you get $$\omega = \cos \frac{2\pi}3 + i \sin \frac{2\pi}3,\ \overline\omega = \omega^2 = \cos \frac{4\pi}3 + i \sin \frac{4\pi}3,\ z = \omega^3 = \cos \frac{6\pi}3 + i \sin \frac{6\pi}3 = 1.$$