I need some information about the Sylow $p$-subgroups, and their normalizers, (specially their sizes), of a finite simple group of Lie type over a finite field (not necessarily algebraic closed) of characteristic $p$.
I would really be appreciate being introduced to a good reference or to be given some information directly.
On
You can find a lot of this information in these monographs by Solomon and Lyons and this book by Jim Humphreys. The article Constructive Sylow theorems for the classical groups, by Mark Stather, should also be very useful to you.
The Sylow $p$-subgroups of finite groups of Lie type in characteristic $p$ are exactly the maximal unipotent subgroups $U$; their normalizers are the Borel subgroups $B$. The collection of Sylow $p$-subgroups forms a space called the flag variety $G/B$ and is actually something you probably studied a little bit back in linear algebra.
Little more detail, little more overview:
In classical groups (GL/SL/PSL, GO/Ω/PΩ, CSp/Sp/PSp, GU/SU/PSU) $U$ is described as a centralizer of a maximal totally singular flag, which is fancy language for “is an upper triangular matrix with 1s on the diagonal.” The Sylow normalizers $B$ are described as the normalizer of a maximal totally singular flag, which is fancy language for “is an upper triangular matrix with nonzeros on the diagonal.” The extra part of the normalizer is called a maximally-split maximal torus $T$, and $B=T\ltimes U$ is a semi-direct product. $T$ is just a fancy way of saying “the diagonal matrices.”
The classical descriptions usually require a combination of an argument that works in all 4 cases, partial arguments for each type, and a few special cases. Many of the other groups of Lie type can be handled in similar ways. Wilson's Finite Simple Groups basically takes this route, and I think is very helpful for getting a grip on a particular group of Lie type.
However, groups of Lie type are usually described using the concept of root subgroups and the Chevalley commutator formula. Other than (a) which linear combination of simple roots are still roots, and (b) the constants appearing in the commutator formula, the description is identical for all groups of Lie type. Thus case by case arguments are usually a bit more rare (often the “good prime” business is where some of those numbers are actually divisible by a prime, so whether they are 0 or nonzero will actually depend on the prime, and so there are some honest to goodness differences in Sylow structure).
Suggested reading:
I learned this material first for groups, but it is significantly easier for Lie algebras. Find an algebraic treatment of Lie algebras of matrices in characteristic 0 (Jacobson's Lie Algebras is fine). Oddly you don't need characteristic $p$ in Lie algebras, since the groups of Lie type (in characteristic $p$) behave much more similarly to Lie algebras in characteristic 0 than they do to Lie algebras in characteristic $p$. Carter's description of unipotent elements is actually pretty understandable after only a chapter or two of Jacobson. It is basically just “wow, eigenvalues are useful!” Lie algebras sounded deeper (to me) than they actually are.
Explanation of root subgroups:
A root subalgebra is just a one-dimensional eigenspace of the torus. In groups, we don't get to use eigenvalues, we have to use "normalizing". A root subgroup is a subgroup minimal amongst the subgroups normalized by $T$. When $G=\operatorname{GL}(n,K)$ and $T$ is the subgroup of all invertible diagonal matrices and $U$ is the subgroup of all upper triangular matrices with $1$s on the diagonal, then the root subgroups $U_{i,j}$ are the subgroups of $U$ in which only $(i,j)$th entry is allowed to be nonzero (other than the diagonals, which still have to be 1). Such a group is isomorphic to the additive group of the field, and the matrix $t$ in $T$ acts as the scalar $t_{i,i} t_{j,j}^{-1}$. For untwisted groups of Lie type, the $U_\alpha$ are similar: it is a subgroup isomorphic to the field, and $t \in T$ acts as the scalar $\alpha(t)$. For twisted and very twisted groups, things are slightly uglier and I suggest ignoring them at first.
So short version: root subgroups are copies of the additive group of the underlying field, and the maximally-split maximal torus normalizes them, each element just acting as a multiplicative scalar.
It is important to note that any subgroup normalized by $T$ (in the algebraically closed case at least) is a product of root subgroups. So the only subgroups that “behave well” are root subgroups.
Chevalley commutator formula:
So we hopefully see that root subgroups build up all the good $p$-subgroups (for instance, their products account for all $p$-subgroups $P$ such that $P=O_p(N_G(P))$, so if you are looking at how local subgroups act on their normal $p$-subgroups, look no further!).
But when we have more than one root subgroup in the mix, the whole “additive group of the field” no longer adequately describes the situation. Sure $U_\alpha$ is a copy of $K$ and $U_\beta$ is a copy of $K$, but how do those copies of $K$ interact?
It turns out they do so in pretty simple ways. For instance in $\operatorname{GL}(3,K)$, we have $U_{1,2}=\{ x(t) : t \in K\}$, $U_{2,3}=\{ y(t) : t \in K \}$, and $U_{1,3} = \{ z(t) : t \in K \}$ are the root subgroups, where $$x(t) = \begin{bmatrix} 1 & t & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}, \quad y(t) = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & t \\ 0 & 0 & 1\end{bmatrix}, \quad z(t) = \begin{bmatrix} 1 & 0 & t \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}$$
The root subgroups $U_{1,2}$ and $U_{2,3}$ are called the simple root subgroups because they generate the rest of $U$ themselves. It is not hard to check the following rules (the Chevalley commutator relations for $\operatorname{GL}_3$): $$\begin{array}{rcl} y(t) \cdot x(s) &=& x(s) \cdot y(t) \cdot z(-st) \\ z(t) \cdot x(s) &=& x(s) \cdot z(t) \\ z(t) \cdot y(s) &=& y(s) \cdot z(t) \\ \end{array}$$ In general, the commutator of two root subgroups is contained in the root subgroups you get from adding them together or even taking linear combinations (way clearer in the Lie algebra case, BTW). $\{1,2\} + \{2,3\} = \{1,3\}$ in root-land. The way you combine the $s$ and the $t$ is always about like that, $c s^a t^b$ for (very small) numbers $a,b,c$.
Suggested reading 2:
So if this made any sense at all (especially after learning about $\mathfrak{su}_3$), then Carter's book Simple Groups of Lie Type is a good place to read the same thing, written more clearly. I've already mentioned Malle–Testerman's Linear Algebraic Groups and Groups of Lie Type which can help you find general principles more easily than Carter's book, but which takes quite a bit longer to get to that point (the gist, by the way, is that finite groups are way harder than a two step process of figuring things out in algebraically closed land, and then converting back down to finite fields; however, it took me a long time to believe that).