I read the book Invariant Theory by T.A. Springer.
There is the following definition of a reductive group:
Definition. A linear algebraic group $G$ is called reductive if for any rational representation $\rho:g\rightarrow GL(V)$ and any $w \in W-\{0\}$ such that $\rho (G)w=w$ there exists $f \in S(W)^G $ with $f(0)=0, f(w)\neq 0$.
The author proposes to prove that $G$ is reductive iff the following holds
Assume that $G$ operates as a group of linear automorphisms of the polynomial algebra $k(T_1, \ldots, T_m)$ such that $$g \cdot T_i= \sum_{j=1}^m x_{j i}(g)T_j,$$ and that the presentation $G\rightarrow GL_m(k)$ with $g\rightarrow (x_{ij}(g))$ is rational representation.
There is a hint: take for $W$ the dual space of $kT_1+kT_2+ \cdots +kT_m$.
I don't understand what is $x_{i,j}$ and I cannot begin the proof.
Let $w_1, ... , w_n$ be a basis for $W$. One can identify $W$ with $k^n$ (where $k$ is your field), and $\textrm{GL}(W)$ with $\textrm{GL}_n$. Therefore you can think of $\textrm{GL}(W)$ as a linear algebraic group.
By definition, a rational representation of $G$ is a homomorphism $\phi:G \rightarrow \textrm{GL}_n$ which is also a morphism of varieties. In particular, if $x_{ij}: \textrm{GL}_n \rightarrow k$ is the morphism of varieties which sends any invertible matrix to its $ij$th coordinate, then the composition $x_{ij} \circ \phi: G \rightarrow k$ is a morphism of varieties, i.e. an element of the coordinate ring $k[G]$. By abuse of notation, write $x_{ij}(g)$ instead of $x_{ij}(\phi(g))$. Therefore, $\phi: G \rightarrow \textrm{GL}_n$ can be described by the formula $$\phi(g) = (x_{ij}(g))$$
Now identify $\textrm{GL}_n$ with $\textrm{GL}(W)$; each matrix $\phi(g) \in \textrm{GL}_n$ is really a $k$-vector space isomorphism $W \rightarrow W$, defined on basis elements by $$\phi(g)(w_j) = \sum\limits_{i=1}^n x_{ij}(g)w_i$$
Thus, a rational representation of $\textrm{GL}(W)$ is a collection of morphisms of varieties $x_{ij}: G \rightarrow k$, such that the matrix $(x_{ij}(g))$ is invertible for all $g \in G$, and such that for some basis $w_1, ... , w_n$ of $G$, the formula for $\phi(g)(w_j)$ holds for all $j$. From here it follows that $g \mapsto \phi(g)$ is a group homomorphism $G \rightarrow \textrm{GL}(W)$.
On the other hand, suppose you identify $W$ with the $k$-vector space spanned by $T_1, ... , T_n$ in the field $k(T_1, ... ,T_n)$. Then for each $g \in G$, the same formula, $$\phi(g)T_j = \sum\limits_{i=1}^n x_{ij}(g)T_i$$ extends uniquely to a $k$-algebra isomorphism of $k(T_1, ... , T_n)$, and the map $g \mapsto \phi(g)$ is a group homomorphism $G \rightarrow \textrm{Aut}_{\textrm{$k$-alg}}k(T_1, ... T_n)$