My question about $PGL(2,q^2)$ has 2 related parts:
1) I am interested in learning about the structure of the group $G:=PGL(2,q^2)$, where $q$ is a prime power. Particularly in the cosets of $PGL(2,q)$ in $PGL(2,q^2)$.
I would like it if you could recommend some introductory references to start learning about this group and these cosets.
2) I would like to be able to enumerate all $\Theta(q^3)$ distinct elements in the coset $PGL(2,q)$ in $PGL(2,q^2)$, without repetition.
I have seen that there are $\Theta(q^3)$ distinct representatives (1 for each coset). My problem is that so far I haven't been able to enumerate them without repetition, so I end up doing more work by enumerating $\Theta(q^4)$ elements instead.
So far my enumeration strategy goes like this:
-Given an element $m_1:=\begin{pmatrix} a & b \\ c & d\end{pmatrix} \in G$, with $a,d \neq 0$, I can perform elementary row operations and transform it into the element: $m_2:=\begin{pmatrix} 1 & \bar{b} \\ \bar{c} & 1\end{pmatrix} \in G$, such that $m_1$ and $m_2$ are equivalent in G.
-Under the assumption that $a,d\neq 0$ I try every possible $\bar{b},\bar{c} \in F_{q^2}$, so I list $\Theta(q^4)$ elements. Then I handle the cases where $a$ or $d$ are $0$.
I have tried other row operations, but still can't enumerate the elements of $G$ without repetitions.
First, as noted in comments, the subgroup is not normal, so that coset space is not a group.
I think there is no universal approach to such questions... but especially in some small cases an approach such as you started does succeed.
Let me first address the slightly different question $GL(2,q^2)/GL(2,q)$, and the original question.
Edit: Thanks to Jack Schmidt for pointing out that there was something wrong with an earlier version.
Let the larger field be $K$, the smaller $k$, and $K=k\oplus k\cdot \omega$ with $\omega\in K$. It is useful to think of $K$ as a $2$-dimensional vector space over $k$. Thus, in your initial $\pmatrix{a&b\cr c&d}$, view $a,b$ as vectors acted on on the right by the $GL(2,k)$, which amounts to taking all non-zero $k$-linear combinations. Edit: IF $a,b$ are linearly independent over $k$, then, in particular, the vector $1\in K$ is in the $k$-span of $a,b$, so, indeed, you can make the $1,1$ entry $1$. Even if they're not linearly independent over $k$, all you need is that their span include $1$. The original discussion forgot the case that $a,b$ might not include $1$ in their span. For this failure to occur, $a,b$ span a one-dimensional $k$-subspace of $K$, and we can put things in the form $\pmatrix{a&0\cr c&d}$ with $a\not\in k$. Can divide through by $k^\times$ to get $\pmatrix{\beta & 0 \cr * & *}$ with $\beta\in (K^\times-k^\times)/k^\times$. Again, I forgot this case first time around.
EDIT-EDIT: and, the lesson learned is that thinking about that normalization of the $1,1$ entry to $1$ is not the optimal viewpoint. The watershed is whether or not the two top-row entries are linearly independent over $k$ or not.
Then it is important to realize that further action by suitable $\pmatrix{1&*\cr 0&1}$ to put the $1,2$ entry in the $1$-dimensional subspace $k\omega$ of $K$. In effect you're taking away the "rational part". If the $1,2$ entry is non-zero in $k\omega$, further adjust it to make it exactly $\omega$.
EDIT: Thus, we have $\pmatrix{\beta & 0\cr *&*}$, with $\beta \in K^\times/k^\times$, and $\pmatrix{1&\omega\cr*&*}$.
The distinction is in the isotropy _subgroups_ in $GL(2,k)$ of the two types of row vectors $(\beta,0)$ (EDITed!) and and $(1,\omega)$, that is, the subgroups that fix them. The first (EDIT: with $\beta$ version, too) has isotropy subgroup $A=\pmatrix{1&0\cr *&*}$, and for $(1,\omega)$ is the trivial subgroup $\{\pmatrix{1& 0\cr 0&1}\}$.
Thus, in the latter case, no further adjustment of the lower row is possible, so there are irredundant representatives of the form $\pmatrix{1 & \omega \cr c' & d'}$ for all $c',d'\in K$ such that the determinant is not $0$, which is the constraint $c'\omega\not=d'$. This gives $(q^2)^2-(q^2)$ such representatives.
In the case $\pmatrix{\beta &0\cr c'&d'}$ (EDITed) further adjustment of the lower row by the isotropy group of the top row is possible: $k$-multiples of $d'$ can be subtracted from $c'$, and $d'$ can be adjusted by a scalar from $k^\times$. Perhaps there are not any obvious canonical choices, but for each $d'$ in some choices of ${q^2-1\over q-1}=q+1$ representatives for $K^\times/k^\times$, the corresponding irredundant $c'$ elements are exactly $k\omega\cdot d'$.
Thus, for the $GL$ groups, if I didn't make a mistake (and I did, first time) there are $(q^4-q^2)+(q+1)(q+1)q=q^4+q^3+q^2+q$ coset representatives. (NOTE: new factor of $q+1$.)
To ask about $PGL(2,K)/PGL(2,k)$, repeat the same general procedure, but now the isotropy groups are slightly larger, since we are asking when a right multiplication by $GL(2,k)$ on $(\beta,1)$'s, and $(1,\omega)$ can be cancelled out by left multiplication by a scalar in $K^\times$.
In the case $(\beta,0)$, the isotropy subgroup is $P=\pmatrix{*&0\cr *&*}$. This allows more adjustments to the lower row than before, since now we can adjust $c'$ further by scalars. Thus, $d'\in K^\times/k^\times$ and $c'=0$ or $\omega d'$.
In the case of $(1,\omega)$, the isotropy group becomes a copy of $K^\times$ imbedded in $GL(2,k)$, namely, something like $\pmatrix{a&b \cr \Delta b & a}$ at least for {\it odd} $q$, taking $\omega=\sqrt{\Delta}$. We can interpret this as saying that we have license to adjust the lower row in $\pmatrix{1&\omega\cr c'&d'}$ by any scalar multiplication, obtaining either $\pmatrix{1&\omega\cr c'&1}$ with $\omega c'\not=1$, or $\pmatrix{1&\omega\cr 1&0}$.