We have this definition:
I didn't understand why in this definition we just define $F^A$ to be
$$F^A(X_0:X_1:X_2)=F\bigg((X_0:X_1:X_2)A\bigg)$$
$F^A$ is not the composition $F\circ c$?
Why complicate things? the definition I stated is suppose to have the same effect in the curve.
Remark: I don't want to know why this definition is stated so, I'm asking why we use inverse matrices in this definition instead of the matrix itself.
Thanks in advance
Short answer: If one took the definition you propose, $F^A(X) := F(X A)$, then you'd be making a left action of $PGL(3,K)$ on your maps ($(F^A)^B(X) = F^A(XB) = F((XB)A) = F(X(BA)) = F^{BA}(X)$), whereas you had a right action on your domain (and, essentially, a trivial right action on your range).
On the other hand, using the definition given in the passage you transcribed, $F^A(X) := F(X A^{-1})$, you get $(F^A)^B(X) = F^A(XB^{-1}) = F(XB^{-1}A^{-1}) = F(X(AB)^{-1}) = F^{AB}(X)$. Hence, you get a right action of $PGL(3,K)$ on the maps $F$.
There could be a deeper reason to want to stick with all right or all left actions in a given context, I'm not sure, but I do know that I would find it confusing to have to keep flipping back and forth between left and right actions.
Longer answer: For simplicity, let's first get rid of all those extra bells and whistles and just think about maps between sets. In particular, suppose you have two right $G$-sets $S$ and $T$. Consider all the maps of sets you could define between $S$ and $T$, $Map(S,T)$. A natural question to ask is whether you can find a sensible action of $G$ on $Map(S,T)$ coming from the actions of $G$ on $S$ and $T$.
Indeed you can! Given a map $f:S\to T$, you can define a new map $f^g(s) := f(s g^{-1})g$. Let's take a moment to check this satisfies the axioms of a right action of $G$ on $Map(S,T)$:
We first need to check that $(f^g)^h = f^{gh}$. This is an exercise in unpacking definitions. $(f^g)^h(s) = (f^g(s h^{-1}))h = (f\left((s h^{-1})g^{-1}\right)g)h = f(s (gh)^{-1}) (gh) = f^{gh}(s)$.
It is obvious that $f^e = f$.
To make contact with your situation, you can set $S = \mathbb{A}_K^3$, $G = GL(3,K)$, $T = K$, instead of all maps $F$ only consider homogeneous polynomials, and then do whatever mental backflips algebraic geometers do to make sense of the quotient everywhere by $K^*$. (Sorry not to be precise here. My AG is rusty and I'm also not sure precisely what AG framework you're working in. I will let you fill in the details, or perhaps an expert will stop by and help us both out.) Notice that in the setup you have, you don't see any outer right action by $A$, you only see the inner right action by $A^{-1}$, because the action of $GL(3,K)$ on $K$ is taken to be the trivial action.
A similar story can also be given for left actions. One thing I like about that case is that you get to write $^g f(s) := gf(g^{-1} s)$, in other words, the usual notational conventions work in your favor in this case and you could simply write $^g f = gfg^{-1}$ without risk of confusion.