This question might be a bit specific, but I am trying to understand Deligne's proof of Tannaka duality in Thm 2.8 of https://www.jmilne.org/math/xnotes/tc2018.pdf right now and I do not understand how the application of Deligne 3.2 from https://www.jmilne.org/math/Books/DMOS.pdf works or what the statement precisely should be (it is buried in Rmk 3.2.a).
My guess is that the setup should be (taking only one faithful representation) as follows:
$G$ a reductive group, $V$ a faithful representation of $G$, $H$ a subgroup of $G$ and $H^\prime$ the subgroup of $G$ fixing all tensors fixed by $H$ that occur in subquotients of finite tensor products of finite sums of $V$. If additionally either $H$ is reductive or $\operatorname{Hom}(G,\mathbb{G}_m)\to\operatorname{Hom}(H,\mathbb{G}_m)$ is surjective then $H=H^\prime$ holds.
In my opinion, this is now applied to the following setup: $A$ an affine group scheme, $V$ a f.d. representation of $A$, denote by $A_V$ the image of $A$ in $\operatorname{GL}_V$.
Then the theorem is applied with $G=\operatorname{GL}_V$, $V=V$ and $H=A_V$. But subgroups of $\operatorname{GL}_V$ in general satisfy neither of the conditions given above and of course we can just take a counterexample to the conditions as $A$.
What am I missing? If somebody with more knowledge on group schemes or representation theory than me could look over this and either point out my mistake or just give general advice on how to solve this, I'd be very thankful.
(Also as far as I know this whole argumentation can be avoided by hitting the problem long enough with category theory, but I want to understand this approach as well)
Let me adopt the notation and setting of Deligne (1982). First, here is an elaboration of Remark 3.2(a):
Lemma: Let $H'$ be the subgroup of $G$ fixing all elements fixed by $H$ in any representation of $G$ (not only representations of the form $T^{m,n}$). Then $H'=H$.
Proof: Obviously $H \subset H'$; conversely, $H$ equals the stabilizer of a line in some such representation, which $H'$ also stabilizes.
Note that this does not require $H$ to be, say, reductive. The way that reductivity was used in the proof of 3.1(c) was to get $H$ as the stabilizer of a "tensor", i.e. an element in a representation of the specific form $T^{m,n}$.
Now, in the proof of Prop 2.8 in Tannakian Categories, we take $H=G_X$ and $H'=\underline{\text{Aut}}^{\otimes}(\omega | C_X)$. This is legitimate because every representation of $\text{GL}_X$ (which is now playing the role of the "ambient" group $G$ in the lemma above) is isomorphic to an object of the category $C_X$. Indeed, by definition $X$ is a faithful representation of $\text{GL}_X$.
I hope this addresses your concerns; if not let me know and I will elaborate further.