I know how to define a group when we have its presentation in GAP by using FreeGroup command over the generators and then taking quotient over the relators, however what if I only have the group as a set, for instance let's say $O$ is the ring of integers of the number field $\mathbb{Q}(\sqrt{-3})$, then I am interested in definig the follwing group in GAP.
$$G:=\{\begin{bmatrix} a & b \\ c\sqrt{-3} & d \end{bmatrix} \in PGL(2,O) | a,b,c,d \in O \}$$
Where $PGL(2,O)$ is the projective general linear group over $O$. So basically this is a subgroup of $PGL(2,\mathbb{C})$.
I don't know if it helps but note that $\sqrt{-3}$ is a prime in $O$ and also $O$ is the $\mathbb{Z}$-module generated by $\{ 1,\frac{1}{2}+\frac{\sqrt{-3}}{2} \}$.
How would one feed $G$ into GAP?
Eventually I want to find the presentation or at least the generators of the group if possible.
I would start with $PGL(2,O)$. A paper by Swan (Generators and relations for certain special linear groups, Adv. Math, 1971, Cor. 6.2) gives you a presentation for $GL(2,O)$. Since it is your lucky day, a preprint of mine https://arxiv.org/abs/2110.04359 gives this presentation, and associated matrices in GAP explicitly. I have not tried, but expect it to be not too hard to find extra relations for $PGO$.
Now for your group. I believe it is the stabilizer in $PGO$ of the line spanned by $(0,1)$, modulo $(\sqrt{-3})$. Reduce the matrix generators modulo this ideal (i.e to matrices over $GF(9)$) and calculate the stabilizer in $PGO$ under action by these matrices. It has finite index, and you can use Reidemeister/Schreier (
IsomorphismFpGroup) on this subgroup to find a presentation (including matrix generators).